Validity and reliability in student learning evaluation throughout active methodologies

Main Article Content

María del R. Medina-Díaz
Ada L. Verdejo-Carrión

Abstract

The demands on university education call for changes on teaching strategies and on the evaluation of student learning. Active methodologies are one of these strategies, which facilitate the development of student learning or competencies, through situations or problems close to the real world and to a professional career. These require to rethink, plan and guide teaching as student-centered and to use techniques and tools for collecting valid and reliable information that leads to an appropriate and a fair evaluation of student learning. However, the validity and reliability of the interpretations of the scores or information collected with these tools have not had sufficient attention, according to the literature reviewed. The purpose of this paper is to discuss the validity of the interpretation and reliability of the scores or the information collected through classroom assessment tools in universities. Accordingly to some publications, a set of recommendations is provided with sources of evidence that underpin validity and reliability. At a minimum, it is suggested taking into account evidence related to content validity and to internal consistency of the scores or the information collected, when making judgments and decisions that affect the students. It is concluded that greater prudence is needed in the interpretations and inferences of learning, if there is insufficient validity evidence.

Article Details

Section
MISCELLANEOUS SECTION

References

Alquraan, M.F. 2012. Methods of assessing students’ learning in higher education An analysis of Jordanian college and grading system. Education, Business and Society: Contemporary Middle Eastern Issues, 5(2), 124-133. https://doi.org/10.1108/17537981211251160

Andreu-Andrés, M.A., & Labrador-Piquer, M.J. 2011. Formación del profesorado en metodologías y evaluación. Análisis cualitativo. Revista de Investigación en Educación, 9(2), 236-245. https://bit.ly/2UhFxu0

Angelo, T.A., & Cross, K.P. 1993. Classroom assessment techniques (2da ed.). Jossey-Bass.

American Educational Research Association, American Psychological Association & National Council on Measurement in Education. 2018. Estándares para pruebas educativas y psicológicas (M. Lieve, Trans.). American Educational Research Association. Original work published 2014.

Baartman, L.K.J., Bastianens, T.J., Kirschner, P.A., & Van der Vieuten, C.P.M. 2006. The wheel of competency assessment: Presenting quality criteria for competency assessment programs. Studies in Educational Evaluation, 32 (2), 153-170. https://doi.org/10.1016/j.stueduc.2006.04.006

Banta, T.W., Lund, J.P., Black, K.E., & Oblander, F.W. 1996. Assessment in practice: Putting principles to work on college campuses. Jossey-Bass.

Banta, T.W., & Pike, B.R. 2012. The bottom line: Will faculty use assessment findings? In C. Secolsky & B. Denison (Eds.), Handbook on measurement, assessment and evaluation in higher education (pp. 47-56). Routledge.

Barkley, E.F., & Major, C.H. 2016. Learning assessment techniques: A Handbook for college faculty. Jossey-Bass.

Black, P., & William, D. 1998a. Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102.

Black, P., & William, D. 1998b. Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80, 139-144.

Bearman, M., Dawson, P., Bennett, S., Hall, M., Molloy, E., Boud, D. & Joughin, G. 2017. How university teachers design assessments: A cross disciplinary study. Higher Education, 74 (1), 49-64. https://doi.org/10.1007/s10734-016-0027-7

Bennett, R. E. 2011. Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18, 5-25. https://doi.org/10.1080/0969594X.2010.513678.

Bonnen, S.M. 2013. Validity in classroom assessment: Purposes, properties, and principles. In J.H. McMillan (Ed.), Sage Handbook of Research on Classroom Assessment (pp. 87-106). Sage.

Brookhart, S.M. 2003. Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5-12. https://doi.org/10.1111/j.1745-3992.2003.tb00139.x

Brookhart, S.M. 2004. Assessment theory for college classrooms. New Directions for Teaching and Learning, 100, 5-14. https://doi.org/10.1002/tl.165

Brookhart, S.M. 2007. Expanding views about formative classroom assessment: A review of the literature. En J.H. McMillan (Ed.), Formative classroom assessment: Theory into practice (pp. 43-62). New York: Teacher College Press.

Brown, G.T.L. 2010. The validity of examination essays in higher education: Issues and responses Higher Education Quarterly, 64(3), 276-291.https://doi.org/10.1111/j.1468-2273.2010.00460.x

Brown, G. & Atkins, M. (1988). Effective teaching in higher education. Routledge.

Cizek, G.J. 2009. Reliability and validity of information about student achievement: Comparing large-scale and classroom testing contexts. Theory into Practice, 48(1), 63-71. https://doi.org/10.1080/00405840802577627

Cizek, G.J. 2015. Validating test score and detecting test score use: Different aims, different methods. Assessment in Education: Principle, Policy & Practice, 23(2), 212-225. https://doi.org/10.1080/0969594X.2015.1063479

Davies, M.S., & Taras, M. 2018. Coherence and disparity in assessment literacies among higher education staff. London Review of Education, 16(3), 474-490. https://doi.org/10.18546/LRE.16.3.09

De la Orden Hoz, A. (2011). Reflexiones en torno a las competencias como objeto de evaluación en el ámbito educativo. Revista Electrónica de Investigación Educativa, 13(2), 1-21. https://bit.ly/2QIILUX

Epstein, R.M. 2007. Assessment in medical education. New England Journal of Medicine, 356 (4), 387-396. https://doi.org/10.1056/NEJMra054784

Erwin, T.D. 1991. Assessing student learning and development. Jossey-Bass.

Esteve Zarazaga, J.M. 2007. Un examen a la cultura escolar. Avances en Supervisión Educativa, (7). https://bit.ly/3ajVMfN

Fernández March, A. 2006. Metodologías activas para la formación de competencias. Educatio Siglo XXI, 24, 36-56. https://bit.ly/39deN20

García-Merino, J. D., Urionabarrenetxea, S., & Bañales-Mallo, A. 2016. Cambios en metodologías docente y de evaluación: ¿Mejoran el rendimiento del alumnado universitario? Revista Electrónica de Investigación Educativa, 18(3), 1-18. https://bit.ly/2WII8yE

Gil Flores, J. 2005. Valoraciones del alumnado universitario sobre las pruebas objetivas. Revista de Investigación Educativa, 23(1), 259-277. https://bit.ly/2wE6z5B

Gilles, J.L., Detroz, P., & Blais, J.G. 2011. An international online survey of the practices and perceptions of higher education professors with respect to the assessment of learning in the classroom. Assessment & Evaluation in Higher Education, 36(6), 719-733. https://doi.org/10.1080/02602938.2010.484880

Gipps, C.V. 1994. Beyond testing: Towards a theory of educational assessment. Falmer Press.

Goñi-Zabala, J.M. (2005). El espacio europeo de educación superior, un reto para la universidad. Ediciones Octaedro.

Goubeaud, K. 2010. How is science learning assessed at the postsecondary level? Assessment and grading practices in college Biology, Chemistry and Physics. Journal of Science Education and Technology, 19 (3), 237-245. https://doi.org/10.1007/s10956-009-9196-9

Goubeaud, K., & Yan, W. 2004. Teachers educators teaching methods, assessments, and grading: A comparison of higher education faculty’s instructional practices. The Teacher Educator, 40 (1), 1-16. https://doi.org/10.1080/08878730409555348

Haladyna, T.M. 1997. Writing test items to evaluate higher order thinking. Allyn & Bacon.

Hernández, R. (2012). Does continuous assessment in higher education support student learning? Higher Education, 64(4), 489-502. : https://doi.org/10.1007/s10734-012-9506-7

Huba, M.E., & Freed, J.E. 2000. Learned-centered assessment on college campuses. Allyn & Bacon.

Hortigüela-Alcalá, D., Pérez-Pueyo, A., & López-Pastor, V. 2015. Implicación y regulación del trabajo del alumnado en los sistemas de evaluación formativa en educación superior. Revista Electrónica de Investigación y Evaluación Educativa, 21(1), ME6. https://doi.org/10.7203/relieve.21.1.5171

Jacobs, L.C., & Chase, C.I. 1992. Developing and using tests effectively: A guide to faculty. Jossey-Bass.

Joint Committee on Standards for Educational Evaluation. 2015. Classroom assessment standards: Practices for PreK-12 teachers. Sage.

Krzykowski, L., & Kinser, K. 2014. Transparency in student learning assessment. Change, 46(3), 67-73 https://doi.org/10.1080/00091383.2014.905428

López-Pastor, V., & Sicilia-Camacho, A. 2016. Formative and shared assessment in higher education. Lessons learned and challenges for the future. Assessment & Evaluation in Higher Education, 42 (1), 77–97, http://dx.doi.org/10.1080/02602938.2015.1083535

Mateo, J., & Martínez, F. 2008. Medición y evaluación educativa. La Muralla.

McClarty, K.L., & Gaertner, M.N. 2015. Measuring mastery: Best practices for assessment in competency-based education. American Enterprise Institute for Public Policy Research. https://bit.ly/2MN1m04

McMillan, J.H. 1997. Classroom assessment: Principles and practice for effective instruction. Allyn and Bacon.

McMillan, J.H. 2008. Assessment essentials for standard-based education (2da ed.). Corwin Press.

Medina-Díaz, M., & Verdejo-Carrión, A.L. 2019. Evaluación del aprendizaje estudiantil (6ta ed.). Autoras.

Medina-Gual, L. 2013. La evaluación en el aula: Reflexiones sobre sus propósitos, validez y confiabilidad. Revista Electrónica de Investigación Educativa, 15(2), 34-50. https://bit.ly/2xm8HiH

Messick, S. 1989. Validity. En R.L. Linn (Ed.), Educational measurement (3ra ed., pp. 13-103). Macmillan.

Moss, P.A. 2003. Reconceptualizing validity for classroom assessment. Educational Measurement: Issues and Practice, 22(4), 13-25. https://doi.org/10.1111/j.1745-3992.2003.tb00140.x

Muñiz, J., & Fonseca-Pedrero, E. 2008. Construcción de instrumentos de medida para la evaluación universitaria. Revista de Investigación en Educación, (5), 13-25. https://bit.ly/39izB8f

Newble, D., & Cannon, R. 1991. A handbook for teachers in university and colleges. Kogan Page.

Nitko, A.J., & Brookhart, S.M. 2011. Educational assessment of students (6ta ed.). Pearson.

Olmos-Miguelañez, S., & Rodríguez-Conde, M.J. 2010. Diseño del proceso de evaluación de los estudiantes universitarios españoles: ¿Respondiendo a una evaluación por competencias en el espacio europeo y Educación Superior? Revista Iberoamericana de Educación, 53(1). https://bit.ly/2JsjM4B

Palomba, C.A., & Banta, T.W. 1999. Assessment essentials: Planning, implementing, and improving assessment in higher education. Jossey-Bass.

Pereira, D.R., & Flores, M.A. 2016. Conceptions and practices of assessment in higher education: A study of Portuguese university teachers. Revista Iberoamericana de Evaluación Educativa, 9(1), 9-29. http://dx.doi.org/10.15366/riee2016.9.1.001

Popham, W.J. 2005. Classroom assessment: What teachers need to know (4ta ed.). Pearson.

Poskanzer, S.G. 2002. Higher education law: The faculty. John Hopkins University Press.

Pozo, J.I., & Pérez-Echevarría, M. del R. 2009. (Coords.), Psicología del aprendizaje universitario: La formación de competencias. Morata.

Rawlusyk, P.E. 2018. Assessment in higher education and student learning. Journal of Instructional Pedagogies, 21, 1-34. https://bit.ly/2vPtbzT

Reddy, Y.M., & Andrade, H. 2010. A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435-448. https://doi.org/10.1080/02602930902862859

Ricoy, M.C., & Fernández-Rodríguez, J. 2013. La percepción que tienen los estudiantes universitarios sobre la evaluación: Un estudio de caso. Educación XXI, 16(2), 321-342. https://doi.org/10.5944/educxx1.2.16.10344.

Selke, M.J.G. 2013. Rubric assessment goes to college. Rowman & Littlefield.

Smith, J.K. 2003. Reconsidering reliability in classroom assessment and grading. Educational Measurement: Issues and Practice, 22(4), 26-33. https://doi.org/10.1111/j.1745-3992.2003.tb00141.x

Stemler, S.E. 2004. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9, Article 4. https://doi.org/10.7275/96jp-xz07

Stiggins, R. 2017. The perfect assessment system. Association for Supervision and Curriculum Development.

Suskie, L. 2009. Assessing student learning: A common sense guide (2da ed.). Jossey-Bass.

Taylor, C.S., & Nolen, S. B. (2005). Classroom assessment: Supporting teaching and learning in real classrooms. Pearson.

UNESCO. 1998. La educación superior en el siglo XXI: Visión y acción. Conferencia mundial sobre la Educación Superior, Paris. https://bit.ly/33Qb4GG

Van der Schaaf, M., Baartman, L., & Prins, F. 2012. Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios. Assessment & Evaluation in Higher Education, 37(7), 847-860. https://doi.org/10.1080/02602938.2011.576312

Voorhees, R.A. 2001. Competency-based learning models: A necessary future. New Directions for Institutional Research, 110, 5-13. https://doi.org/10.1002/ir.7

Webber, K. 2012. The use of learner-centered assessment in US Colleges and Universities. Research in Higher Education, 53(2), 201-228. https://doi.org/10.1007/s11162-0119245-0.

Weimer, M. (Ed.). 2013. Grading strategies for the college classroom: A collection of articles for faculty. Magna Publications.

Wolf, K., Dunlap, J., & Stevens, E. 2012. Ten things every professor should know about assessment. The Journal of Effective Teaching, 12(2), 65-79. https://bit.ly/2UBD6Bh

Yükselii, H.S., & Gündüz, N. 2017. Formative and summative assessment in higher education: Opinions and practices of instructors. European Journal of Education Studies, 3(8), 336-356. https://doi.org/10.5281/zenodo.832999