Revisitando as revisões sistemáticas numa perspetiva metodológica
DOI:
https://doi.org/10.30827/relieve.v29i1.27758Palavras-chave:
Síntese, mixed methods, revisões agregativas e configurativas, qualidade metodológica, diagrama PRISMAResumo
A rápida expansão das revisões sistemáticas nos últimos anos levou a um crescimento que, muitas vezes, não teve o controlo necessário. Foram efetuadas inúmeras revisões sistemáticas em praticamente todos os domínios e subdomínios do conhecimento, influenciadas pelo movimento da prática baseada na evidência e na facilidade de acesso a um vasto leque de fontes bibliográficas. No entanto, é necessário reposicioná-las numa perspetiva metodológica, e por diversas razões, tais como estar atento ao uso nem sempre correto da terminologia e dos conceitos processuais, definir os objetivos e a abordagem em termos de estratégia, propor uma estrutura adequada, dar às revisões sistemáticas uma amplitude, profundidade e extensão ideais, e avaliar a qualidade metodológica dos documentos primários, que, atualmente, recomendamos dimensionar a partir de uma abordagem de mixed methods considerada como um continuum, em oposição à polarização anterior nas vertentes qualitativa e quantitativa.
Downloads
Referências
Abrami, P.C., Borokhovski, E., Bernard, R.M., Wade, C.A., Tamim, R., Persson, T., & Surkes, M.A. (2010). Issues in conducting and disseminating brief reviews of evidence. Evidence & Policy: A Journal of Research, Debate and Practice, 6, 371–389. https://doi.org/10.1332/174426410X524866
Ailinger, R. (2003). Contribution of qualitative evidence to evidence based practice in nursing. Revista Latinoamericana de Enfermagem, 11(3), 275-279. https://doi.org/10.1590/S0104-11692003000300002
Albrecht, B.M., Foettinger, L., & Bammann, K. (2021). Men’s sheds as community based health promotion for men aged 50 plus: protocol for a mixed-methods systematic review. Systematic Reviews, 10, 215. https://doi.org/10.1186/s13643-021-01762-x
Allers, K., Hoffmann, F., Mathes, T., & Pieper, D. (2018). Systematic reviews with published protocols compared to those without: more effort, older search. Journal of Clinical Epidemiology, 95,102–10. https://doi.org/10.1016/j.jclinepi.2017.12.005
Allwood, C.M. (2012). The distinction between qualitative and quantitative research methods is problematic. Quality & Quantity, 46, 1417-1429. https://doi.org/10.1007/s11135-011-9455-8
Anguera, M.T. (2022). Profundizando en el análisis en mixed methods: Integración de elementos cualitativos y cuantitativos en el marco de la observación sistemática del comportamiento. Discurso Doctorado Honoris Causa. Tenerife: Universidad de La Laguna.
Anguera, M.T., Blanco-Villaseñor, A., Losada, J.L., Sánchez-Algarra, P., & Onwuegbuzie, A.J. (2018). Revisiting the difference between mixed methods and multimethods: Is it all in the name? Quality & Quantity, 52, 2757-2770. https://doi.org/10.1007/s11135-018-0700-2
Anguera, M.T., Jonsson, G.K., Escolano-Pérez, E., Sánchez-López, C.R., Losada, J.L., & Portell, M. (2023). T-pattern detection in the scientific literature of this century: A systematic review. Frontiers in Psychology, 14,1085980. https://doi.org/10.3389/fpsyg.2023.1085980
Aromataris, E., & Munn, Z. (Eds.) (2021). JBI Manual for Evidence Synthesis. https://doi.org/10.46658/JBIMES-20-01
Backman, C., Crick, M., Cho-Young, D., Scharf, M., & Shea, B. (2018). What is the impact of sensory practices on the quality of life of long-term care residents? A mixed-methods systematic review protocol. Systematic Reviews, 7, 115. https://doi.org/10.1186/s13643-018-0783-9
Barbour, R.S., & Barbour, M. (2003). Evaluating and synthesizing qualitative research: The need to develop a distinctive approach. Journal of Evaluation in Clinical Practice, 9, 179-186. https://doi.org/10.1046/j.1365-2753.2003.00371.x
Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC Medical Research Methodology, 9,59.
Boaz, A., Ashby, D., Denyer, D., Egan, M., Harden, A., Jones, D. R., . . ., & Tranfield, D. (2006). A multitude of syntheses: A comparison of five approaches from diverse policy fields. Evidence & Policy, 2, 479-502. https://doi.org/10.1332/174426406778881755
Bohlin, I. (2012). Formalizing syntheses of medical knowledge: The rise of meta-analysis and systematic reviews. Perspectives on Science, 20(3), 273-309. https://doi.org/10.1162/POSC_a_00075
Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2011). An international registry of systematic-review protocols. Lancet, 377, 108-109. https://doi.org/10.1016/S0140-6736(10)60903-8
Brignardello-Petersen, R., Ioannidis, J.P.A., Tomlimson, G., & Guyatt, G. (2015). Surprising results of randomized trials. In G. Guyatt, D. Rennie, M.O. Meade, and D.J. Cook (Eds.), Users’ guide to the medical literature. A manual for evidence-based clinical practice, 3rd ed. McGraw-Hill.
Broome, M.E. (1993). Integrative literature reviews for the development of concepts. In B.L. Rodgers, and K.A. Knafl (Eds.), Concept development in nursing (pp. 231-250). W.B. Saunders Co.
Brown, J.V.E., Ajjan, R., Siddiqi, N., & Coventry, P.A. (2022). Acceptability and feasibility of continuous glucosa monitoring in people with diabetes: protocol for a mixed-methods systematic review of quanitative and qualitative evidence. Systematic Reviews, 11, 263. https://doi.org/10.1186/s13643-022-02126-9
Bunn, F., Trivedi, D., Alderson, P., Hamilton, L., Martin, A., Pinkney, E., & Iliffe, S. (2015). The impact of Cochrane Reviews: A mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. Health Technology Assessment, 19(28), 1-100. https://doi.org/10.3310/hta19280
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2, 40. https://doi.org/10.1186/1748-5908-2-40
Collins, K. & O’Cathain, A. (2009). Ten points about mixed methods research to be considered by the novice researcher. International Journal of Multiple Research Approaches, 3, 2-7. https://doi.org/10.5172/mra.455.3.1.2
Chacón-Moscoso, S., Sanduvete-Chaves, S., Lozano-Lozano, J.A., Portell, M., & Anguera, M.T. (2021). From randomized control trial to mixed methods: A practical framework for program evaluation based on methodological quality. Anales de Psicología / Annals of Psychology 37(3), 599-608. https://doi.org/10.6018/analesps.470021
Chalmers, I., Hedges, L., & Cooper, H. (2002). A brief history of research synthesis. Evaluation of Health Professions, 25, 12–37. https://doi.org/10.1177/0163278702025001003
Chang, Y., Voils, C. I., Sandelowski, M., & Crandell, J. L. (2009). Transforming verbal counts in reports of qualitative descriptive studies into numbers. Western Journal of Nursing Research, 31(7), 837–852. https://doi.org/10.1177/0193945909334434
Chicoine, G., Côté, J., Pepin, J., Fontaine, G., Maheu-Cadotte, M.-A., Hong, Q.N., Rouleau, G., Ziegler, D., & Jutras-Aswad, D. (2021). Effectiveness and experiencies of the Extension for Community Healthcare Outcomes (ECHO) Model in developing competencies among healthcare professionals: a mixed methods systematic review protocol. Systematic Reviews, 10, 313. https://doi.org/10.1186/s13643-021-01832-0
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005). Synthesizing qualitative and quantitative evidence: A review of possible methods. Journal of Health Services Research & Policy, 10, 45-53. https://doi.org/10.1258/1355819052801804
Eisner, E.W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. MacMillan.
Fàbregues, S., Molina-Azorín, J.F., & Fetters, M.D. (2021). Virtual special issue on “Quality in Mixed Methods Research”. Journal of Mixed Methods Research, 15(2), 146-151. https://doi.org/10.1177/15586898211001974
Fàbregues, S., Paré, M. H., & Meneses, J. (2019). Operationalizing and conceptualizing quality in mixed methods research: A multiple case study of the disciplines of education, nursing, psychology, and sociology. Journal of Mixed Methods Research, 13(4), 424-445. https://doi.org/10.1177/155868981 7751774
Glasziou, P.P., & Sanders, S.L. (2002). Investigating causes of heterogeneity in systematic reviews. Statistics in Medicine, 21, 1503-1511. https://doi.org/10.1002/sim.1183
Greckhamer, T., & Koro-Ljungberg, M. (2005). The erosion of a method: Examples from grounded theory. International Journal of Qualitative Studies in Education, 18, 729-750. https://doi.org/10.1080/09518390500298204
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005). Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Social Science Medicine, 61,417–430. https://doi.org/10.1016/j.socscimed.2004.12.001
Gough, D. (2007, January). Dimensions of difference in evidence reviews. National Centre for Research Methods Meeting. Manchester: EPPI-Centre. http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=1919
Gough, D., & Thomas, J. (2012). Commonality and diversity in reviews. In D. Gough, S. Oliver, & J. Thomas (Eds.), Introduction to Systematic Reviews (pp. 35-65). Sage.
Gough, D. (2015). Qualitative and mixed methods in systematic reviews. Systematic Reviews, 4, 181. https://doi.org/10.1186/s13643-015-0151-y
Gough, D., Oliver, S., Newman, M., & Bird, K. (2009). Transparency in planning, warranting and interpreting research. Teaching and Learning Research Briefing 78. London: Teaching and Learning Research Programme.
Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28. https://doi.org/10.1186/2046-4053-1-28
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26, 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine, 61, 417-430. https://doi.org/10.1016/j.socscimed.2004.12.001
Harden, A., & Gough, D. (2012). Quality and relevance appraisal. In D. Gough, S. Oliver and J. Thomas (Eds.), Introduction to Systematic Reviews (pp. 153-178). Sage.
Harden, A., & Thomas, J. (2005). Methodological issues in combining diverse study types in systematic reviews. International Journal of Social Research Methodology, 8, 257-271. https://doi.org/10.1080/13645570500155078
Hawker, S., Payne, S., Kerr, C., Hardey, M., & Powell, J. (2002). Appraising the evidence: Reviewing the disparate data systematically. Qualitative Health Research, 12, 1284-1299. https://doi.org/10.1177/1049732302238251
Heyvaert, M., Hannes, K., & Onghena, P. (2016). Using mixed methods research synthesis for literature reviews: The mixed methods research synthesis approach. Sage. https://doi.org/10.4135/9781506333243
Heyvaert, M., Maes, B., & Onghena, P. (2013). Mixed methods research synthesis: Definition, framework, and potential. Quality & Quantity, 47, 659-676. https://doi.org/10.1007/s11135-011-9538-6
Hoffmann, F., Allers, K., Rombey, T., Helbach, J., Hoffmann, A., Mathes, T., et al. (2021). Nearly 80 systematic reviews were published each day: observational study on trends in epidemiology and reporting over the years 2000-2019. Journal of Clinical Epidemiology, 138, 1–11. https://doi.org/10.1016/j.jclinepi.2021.05.022
Hong, Q.M., & Pluye, P. (2019). A conceptual framework for critical appraisal in systrematic mixed studies reviews. Journal of Mixed Methods Research, 13(4), 446-460. https://doi.org/10.1177/1558689818770058
Hong, Q.M., Pluye, P., Bujold, M., & Wassef, M. (2017). Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Systematic Reviews, 6:61. https://doi.org/10.1186/s13643-017-0454-2
JBI Manual for Evidence Synthesis (2021). CRICOS Provider Number 00123M. jbi.global
Johnson, E.E., O’Keefe, H., Sutton, A., & Marshall, C. (2022). The Systematic Review Toolbox: keeping up to date with tools to support evidence sythesis. Systematic Reviews, 11, 258. https://doi.org/10.1186/s13643-022-02122-z
Johnson, R.B., & Onwuegbuzie, A.J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33, 14-26. https://doi.org/10.3102/0013189X033007014
Johnson, R.B., Onwuegbuzie, A.J., & Turner, L.A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112–33. https://doi.org/10.1177/1558689806298224
Johnstone, A., McCrorie, P., Cordovil, R., Fjørtoft, I., Livonen, S., Jidovtseff, B., Lopes, F., Reilly, J.J., Thomson, H., Wells, V., & Martin, A. (2020). Nature-based early childhood education for child health, wellbeing and development: a mixed-methods systematic review protocol. Systematic Reviews, 9, 226. https://doi.org/10.1186/s13643-020-01489-1
Jones, K. (2004). Mission drift in qualitative research, or moving toward a systematic review of qualitative studies, moving back to a more systematic narrative review. The Qualitative Report, 9(1), 95-112.
Kirkham, J.J., Altman, D.G., & Williamson, P.R. (2010). Bias due to changes in specified outcomes during the systematic review process. PLoS One, 5, e9810. https://doi.org/10.1371/journal.pone.0009810
Krleza-Jeric, K., Chan, A.W., Dickersin, K., Sim, I., Grimshaw, J., & Gluud, C. (2005). Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ, 330(7497), 956–8. https://doi.org/10.1136/bmj.330.7497.956
Leslie, L., Cherry, R.F., Mulla, A., Abbott, J., Furfari, K., Glover, J.J., Hamke, B., Wynia, M.K. (2016). Domains of quality for clinical ethics case consultation: a mixed-method systematic review. Systematic Reviews, 5, 95. https://doi.org/10.1186/s13643-016-0273-x
Lockwood, C., Porrit, K., Munn, Z., Rittenmeyer, L., Salmond, S., Bjerrum, M., Loveday, H., Carrier, J, & Stannard, D. (2020). Systematic reviews of qualitative evidence. In E. Aromataris, and Z. Munn (Eds.), JBI Manual for Evidence Synthesis. JBI. https://doi.org/10.46658/JBIMES-20-03
Lucas, P.J., Baird, J., Arai, L., Law, C., & Roberts, H.M. (2007). Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Medical Research Methodology, 7, 4. Retrieved from http://www.biomedcentral.com/1471-2288/7/4
Mays, N., Pope, C., & Popay, J. (2005). Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. Journal of Health Services Research and Policy, 10 (Suppl. 1), 6-20. https://doi.org/10.1258/1355819054308576
Maxwell, J.A. (2004a). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33, 3-11. https://doi.org/10.3102/0013189X033002003
Maxwell, J.A. (2004b). Using qualitative methods for causal explanation. Field Methods, 16, 243-264. https://doi.org/10.1177/1525822X04266831
Mazevska, D., Pearse, J, & Tierney, S. (2022). Using a theoretical framework to inform implementation of the patient-centred medical home (PCMH) model in primary care: protocol for a mixed-methods systematic review. Systematic Reviews, 11, 249. https://doi.org/10.1186/s13643-022-02132-x
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4,1. https://doi.org/10.1186/2046-4053-4-1
Morse, J.M. (1994). On the crest of a wave? (Editorial). Qualitative Health Research, 4, 139-141. https://doi.org/10.1177/104973239400400201
Munafo, M.R., Hollands, G.J., & Marteau, T.M. (2018). Open science prevents mindless science. BMJ, 363, k4309. https://doi.org/10.1136/bmj.k4309
Newman, I., & Benz, C. R. (1998). Qualitative-quantitative research methodology: Exploring the interactive continuum. Southern Illinois University Press.
Newman, I., & Hitchcock, J. H. (2011). Underlying agreements between quantitative and qualitative research: The short and tall of it all. Human Resource Development Review, 10(4), 381-398. https://doi.org/10.1177/1534484311413867
Onwuegbuzie, A. (2012). Putting the MIXED back into quantitative and qualitative research in educational reearch and beyond: Moving towards the ‘radical middle’. International Journal of Multiple Research Approaches, 6, 192-219. https://doi.org/10.5172/mra.2012.6.3.192
Onwuegbuzie, A.J. & Teddie, C. (2003). A framework for analyzing data in mixed methods research. In A. Tashakkori & C. Teddie (Edss), Handbook of mixed metods in social and behavioral research (pp. 351-383). Sage.
Onwuegbuzie, A., & Tashakkori, A. (2015). Utilizing mixed research and evaluation methodology in peace psychology and beyond. In D. Bretherton & S. F. Law (Eds.), Methodologies in peace psychology: Peace research by peaceful means (pp. 115-137). New York, NY: Springer. https://doi.org/10.1007/978-3-319-18395-4_6
Owens, C. M., & Ferron, J. M. (2012). Synthesizing single-case studies: A Monte Carlo examination of a three-level meta-analytic model. Behavior Research Methods, 44(3), 795–805. https://doi.org/10.3758/s13428-011-0180-y
Pace, R., Pluye, P., Bartlett, G., Macaulay, A.C., Salsberg, J., Jagosh, J., et al. (2012). Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. International Journal of Nursing Studies, 49 (1), 47–53. https://doi.org/10.1016/j.ijnurstu.2011.07.002
Page, M.J., Shamseer, L., & Tricco, A.C. (2018). Registration of systematic reviews in PROSPERO: 30,000 records and counting. Systematic Reviews, 7(1), 32. https://doi.org/10.1186/s13643-018-0699-4
Parkhurst, P.E., Lovell, K.L., Sprafka, S.A., & Hodgins, M. (1972). Evaluation of videodisc modules: A mixed methods approach. (ERIC Document Reproduction Service No. ED348014)
Pawson, R. (2006). Evidenced-based policy: A realist perspective. Sage. https://doi.org/10.4135/9781849209120
Pearson, S.A., Taylor, S, Marsden, A., & Yorke, J. (2021). Access to systemic anti-cancer therapies for women with secondary breast cancer – protocol for a mixed methods systematic review. Systematic Reviews, 10, 209. https://doi.org/10.1186/s13643-021-01761-y
PLoS Medicine Editors (2011). Best Practice in Systematic Reviews: The Importance of Protocols and Registration. PLoS Medicine, 8(2), e1001009. https://doi.org/10.1371/journal.pmed.1001009
Pluye, P. (2015). Mixed kinds of evidence: synthesis designs and critical appraisal for systematic mixed studies reviews including qualitative, quantitative and mixed methods studies. Evidence Based Medicine. Letter. https://doi.org/10.1136/ebmed-2014-110158
Pluye, P., Gagnon, M.P., Griffiths, F., Johnson-Lafleur, J. (2009). A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative, and mixed methods primary-level studies in mixed studies reviews. International Journal of Nursing Studies, 46, 529-546. https://doi.org/10.1016/j.ijnurstu.2009.01.009
Pluye, P., & Hong, Q. N. (2014). Combining the power of stories and the power of numbers: Mixed methods research and mixed studies reviews. Annual Review of Public Health, 35, 29-45. https://doi.org/10.1146/annurev-publhealth-032013-182440
Pluye, P., Robert, E., Cargo, M., Bartlett, G., O’Cathain, A., Griffiths, F., et al. (2011). Proposal: A Mixed Methods Appraisal Tool for Systematic Mixed Studies Reviews. http://mixedmethodsapprai-saltoolpublic.pbworks.com
Pound, P., Britten, N., Morgan, M., Yardley, L., Pope, C., Daker-White, G., et al. (2005), Resisting medicines: A sythesis of qualitative studies of medicine taking. Social Science & Medicine, 6, 133-155. https://doi.org/10.1016/j.socscimed.2004.11.063
Preciado, M., Anguera, M.T., Olarte, M., & Lapresa, D. (2019). Observational studies in male elite football: A systematic mixed study review. Frontiers in Psychology, 10, 2077. https://doi.org/10.3389/fpsyg.2019.02077
Preciado, M., Anguera, M.T., Olarte, M., y Lapresa, D. (2021). Revisión Sistemática en Fútbol Sala desde los Mixed Methods. Revista de Psicología del Deporte / Journal of Sports Psychology, 30(1), 75-96.
Queiroga Souto, R., Khanassov, V., Hong, Q.N., Busch, P.L., Vedel, Is., & Pluye, P. (2015). Systematic mixed studies reviews: Updating results on the reliability and efficiency of the mixed methods appraisal tool. International Journal of Nursing Studies, 52, 500-501. https://doi.org/10.1016/j.ijnurstu.2014.08.010
Rana, D., Westrop, S., Germeni, E., McGarty, A., Ells, L., Lally, P., McEwan, M., Melville, C., Harris, L., & Wu, O. (2021). Understanding the effectiveness and underlying mechanisms of lifestyle modification interventions in adults with learning disabilities: protocol for a mixed-methods systematic review. Systematic Reviews, 10, 251. https://doi.org/10.1186/s13643-021-01808-0
Ridenour, C.S., & Newman, I. (2008). Mixed methods research: Exploring the interactive continuum. Southern Illinois University Press.
Sandelowski, M. (2004). Using qualitative research. Qualitative Health Research, 14, 1366-1386. https://doi.org/10.1177/1049732304269672
Sandelowski, M. (2014). Unmixing mixed-methods research. Research in Nursing & Health, 37, 3–8. https://doi.org/10.1002/nur.21570
Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. Springer.
Sandelowski, M, Voils, C. I., & Knafl, G. (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208-222. https://doi.org/10.1177/1558689809334210
Sandelowski, M., Voils, C.I., & Barroso, J. (2006). Defining and designing mixed research synthesis studies. Research in the Schools, 13, 29-40.
Sandelowski, M., Voils, C.J., Leeman, J., & Crandlee, J.L. (2011). Mapping the Mixed Methods-Mixed Research Synthesis Terrain. Journal of Mixed Methods Research, 6(4), 317-331. https://doi.org/10.1177/1558689811427913.
Sarmento, H., Anguera, M.T., Pereira, A., & Araujo, D. (2018). Talent Identification and Development in Male Football: A Systematic Review. Sports Medicine, 48(4), 907-931. https://doi.org/10.1007/s40279-017-0851-7
Silagy, C.A., Middleton, P., & Hopewell, S. (2002). Publishing protocols of systematic reviews: Comparing what was done to what was planned. JAMA, 287, 2831-2834. https://doi.org/10.1001/jama.287.21.2831
Smith, V., Devane, D., Begley, C.M., & Clarke, M. (2011). Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Medical Research Methodology, 11, 15. https://doi.org/10.1186/1471-2288-11-15
Souto, R., Khanassov, V., Pluye, P., Hong, Q.N., Bush, P., Vedel, I. (2014). Systematic mixed studies reviews: reliability testing of the Mixed Methods Appraisal Tool. In: Mixed Methods International Research Association (MMIRA) Conference. Boston, USA. http:// mixedmethodsappraisaltoolpublic.pbworks.com
Stewart, R., & Oliver, S. (2012). Making a difference with systematic reviews. In D. Gough, S. Oliver, and J. Thomas (Eds.), Introduction to Systematic Reviews (pp. 227-244). Sage.
Straus, S., & Moher, D. (2010). Registering systematic reviews. CMAJ, 182, 13-14. https://doi.org/10.1186/1471-2288-11-15
Suri, H., & Clarke, D. (2009). Advancements in research synthesis methods: From a methodologically inclusive perspective. Review of Educational Research, 79(1), 395-430. https://doi.org/10.3102/0034654308326349
Sutton, A.J., Abrams, K.R., Jones, D.R., Sheldon, T.A., & Song, F. (2000). Methods for meta-analysis in medical research. Wiley.
Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social and behavioral research. Sage.
Tashakkori, A., & Teddlie, C. (Eds.). (2010). Handbook of mixed methods in social and behavioral research (2nd ed.). Sage.
Taylor, C., & Coffey, A. (2009). Special issue: Qualitative research and methodological innovation. Qualitative Research, 9, 523-526. https://doi.org/10.1177/1468794109350355
Teddlie, C. & Tashakkori, A. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13, 12-28.
Tennant, J.P., Waldner, F., Jacques, D.C., Masuzzo, P., Collister, L.B., Hartgerink, C.H. (2016). The academic, economic and societal impacts of open access: an evidence-based review. F1000Research, 5, 632. https://doi.org/10.12688/f1000research.8460.3.
Travers, M. (2009). New methods, old problems: A skeptical view of innovation in qualitative research. Qualitative Research, 9, 161-179. https://doi.org/10.1177/1468794108095079
Tricco, A.C., Pham, B., Brehaut, J., Tetroe, J., & Cappelli, M. (2009). An international survey indicated that unpublihed systematic reviews exist. Journal of Clinical Epidemiology, 62, 617-623. https://doi.org/10.1016/j.jclinepi.2008.09.014
Trinder, L. (2000). A critical appraisal of evidence-based practice. In L. Trinder & S. Reynols (Eds.), Evidence-based practice: A critical appraisal (pp. 212-241). Bkackwell Science. https://doi.org/10.1002/9780470699003
Trinder, L. & Reynolds, S. (Eds.) (2000). Evidence-based practice: A critical appraisal. Blackwell Science. https://doi.org/10.1002/9780470699003.ch10
Tronchoni, H., Izquierdo, C., & Anguera, M.T. (2022). A systematic review on lecturing in contemporary university teaching. Frontiers in. Psychology, 13, 971617. https://doi.org/10.3389/fpsyg.2022.971617
Tufanaru, C., Munn, Z., Aromataris, E., Campbell, J., & Hopp, L. (2020). Systematic reviews of effectiveness. In E. Aromataris, and Z. Munn (Eds.), JBI Manual for Evidence Synthesis. JBI. https://doi.org/10.46658/JBIMES-20-04
Van der Braak, K., Ghannad, M., Orelio, C., Heus, P., Damen, J.A.A, Spijker, R., Robinson, K., Lund, H., & Hooft, L. (2022). The score after 10 years of registration of systematic review protocols. Systematic Reviews, 11, 191. https://doi.org/10.1186/s13643-022-02053-9
Vandyk, A.D., Kaluzienski, M., Goldie, C., Stokes, Y., Ross-White, A., Kronick, J., Gilmour, M., MacPhee, C., & Graham, I.D. (2019). Interventions to improve emergency department use for mental health reasons: protocol for a mixed-methods systematic review. Systematic Reviews, 8, 84. https://doi.org/10.1186/s13643-019-1008-6
Vicente-Sáezooft, L. (2022), R., & Martínez-Fuentes, C. (2018). Open science now: a systematic literature review for an integrated definition. Journal of Business Research, 88, 428–36. https://doi.org/10.1016/j.jbusres.2017.12.043
Voils, C.I., Sandelowski, M., Barroso, J., Hasselblad, V. (2008). Making sense of qualitative and quantitative findings in mixed research synthesis studies. Field Methods, 20(1), 3-25. https://doi.org/10.1177/1525822X07307463
Wiafe, E., Mensah, K.B., Mensah, A.B.B., Bandalee, V., & Oosthuizen, F. (2020). The awareness of women on prostate cancer: a mixed-methods systematic review protocol. Systematic Reviews, 9, 253. https://doi.org/10.1186/s13643-020-01513-4
![](https://revistaseug.ugr.es/public/journals/41/submission_27758_22954_coverImage_es_ES.jpg)
Downloads
Publicado
Como Citar
Edição
Secção
Licença
Direitos de Autor (c) 2023 RELIEVE - Revista Electrónica de Investigación y Evaluación Educativa
![Creative Commons License](http://i.creativecommons.org/l/by-nc/4.0/88x31.png)
Este trabalho encontra-se publicado com a Creative Commons Atribuição-NãoComercial 4.0.
Los autores ceden de forma no exclusiva los derechos de explotación de los trabajos publicados a RELIEVE (a los solos efectos de favorecer la difusión de los artículos publicados:firmar contratos de difusión, de integración en bases de datos, etc.) y consienten que se distribuyan bajo la licencia de Creative Commons Reconocimiento-Uso No Comercial 4.0 International (CC-BY-NC 4.0), que permite a terceros el uso de lo publicado siempre que se mencione la autoría de la obra y la fuente de publicación, y se haga uso sin fines comerciales.
Los autores pueden llegar a otros acuerdos contractuales adicionales e independientes, para la distribución no exclusiva de la versión del trabajo publicado en esta revista (por ejemplo, incluyéndolo en un repositorio institucional o publicándolo en un libro), siempre y cuando se cite claramente que la fuente original de publicación es esta revista.
La mera remisión del artículo a RELIEVE supone la aceptación de estas condiciones.