Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education

Semiral Öncü

doi:10.12973/eurasia.2015.1343a

Full Text (PDF)

Semiral Öncü ¹ ^*

More Detail

¹ Uludağ University^* Corresponding Author

Abstract

Background:
Many institutions monitor academic engagement to investigate student achievement and institutional performance. Relying only on self-reports is prone to misjudgment. Peer evaluation through teamwork has the potential to substitute for measuring engagement, which has not been emphasized in the literature. This study examines whether peer evaluation can represent perceived academic engagement with coursework, operationalized as “active learning” and “paying attention.”

Materials and methods:
Data were collected through an online peer evaluation system. Participants were 215 undergraduate and graduate students from three universities. Structural equation modeling (SEM) was used to test hypotheses.

Results:
Results show that peer evaluation is statistically predictive of perceived academic engagement. Peers give high scores to their counterparts conforming to the literature. Students are more engaged than overall benchmarks.

Conclusions:
Results suggest that creating more opportunities for teamwork in the educational environment and efficient ways to incorporate peer evaluation through means such as social networks and online learning management systems are viable venues for improvement.

Keywords

collaboration
peer evaluation
active learning
paying attention
perceived academic engagement

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

EURASIA J Math Sci Tech Ed, Volume 11, Issue 3, 2015, 535-549

https://doi.org/10.12973/eurasia.2015.1343a

Publication date: 26 Apr 2015

Article Views: 2907

Article Downloads: 1501

Open Access References How to cite this article

References

Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M.-T. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education, 53(1), 74-85.
Astin, A. W. (1993). What matters in college? For critical years revisited. San Francisco, CA: Jossey-Bass.
Baker, D. P., Horvath, L., Campion, M., Offermann, L., & Salas, E. (2005). Chapter 7: The ALL Teamwork Framework. In T. S. Murray, Y. Clermont & M. Binkley (Eds.), International Adult Literacy Survey Measuring Adult Literacy and Life Skills: New Frameworks for Assessment - Catalogue no. 89-552-MIE, no. 13 (pp. 229-272). Ottawa, Ontario, Canada: Statistics Canada.
Baxter, J. (2012). Who am I and what keeps me going? Profiling the distance learning student in higher education. International Review of Research in Open and Distance Learning, 13(4), 107–129.
Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural modeling. Sociological Methods and Research, 16(1), 78-117.
Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the linkages. Research in Higher Education, 47(1), 1-32.
Chen, P., Gonyea, R., & Kuh, G. (2008). Learning at a distance: Engaged or not. Innovate: Journal of Online Education, 4(3), 1-8.
Chen, P.-S. D., Lambert, A. D., & Guidry, K. R. (2010). Engaging online learners: The impact of web-based learning technology on college student engagement. Computers & Education, 54(4), 1222-1232.
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-7.
Coates, H. (2005). The value of student engagement for higher education quality assurance. Quality in Higher Education, 11(1), 25-36.
Cooper, J. L., MacGregor, J., Smith, K. A., & Robinson, P. (2000). Implementing small group instruction: Insights from successful practitioners. In J. MacGregor, J. L. Cooper, K. A. Smith. & P. Robinson (Eds.), Strategies for energizing large classes: From small groups to learning communities (Vol. 2000, pp. 63-76). San Francisco, CA: Jossey-Bass.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
De Grez, L., Valcke, M., & Roozen, I. (2012). How effective are self-and peer assessment of oral presentation skills compared with teachers’ assessments? Active Learning in Higher Education, 13(2), 129-142.
Esposto, A., & Weaver, D. (2008). Strengthening teamwork to improve student engagement in managerial economics and strategy. In Higher Education Research and Development Society of Australasia (HERDSA). Rotorua, New Zealand.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of marketing research, 48, 39- 50.
Franker, K. (2011). Wiki Rubric. Retrieved April 4, 2013 from World Wide Web http://www2.uwstout.edu/content/profdev/rubrics/wi kirubric.pdf.
Freeman, M., & McKenzie, J. (2002). SPARK, a confidential web–based template for self and peer assessment of student teamwork: Benefits of evaluating across different subjects. British Journal of Educational Technology, 33(5), 551-569.
Frydenberg, E., Ainley, M., & Russell, V. J. (2005). Schooling issues digest: Student motivation and engagement. In (Vol. 2, pp. 1-16): Australian Government, Department of Education, Science and Training.
Gueldenzoph, L. E., & May, G. L. (2002). Collaborative peer evaluation: Best practices for group member assessments. Business Communication Quarterly, 65(1), 9-20.
Hair, J. F., Jr., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). New Jersey: Prentice-Hall International.
Hake, R. (1998). Interactive-engagement vs. traditional methods: A 6,000-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74.
Harlen, W., & James, M. (1996). Creating a positive impact of assessment on learning. In The Annual Meeting of the American Educational Research Association. New York, NY.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
James, R., Krause, K.-L., & Jennings, C. (2010). The first year experience in Australian universities: Findings from 1994 to 2009. In: Centre for the Study of Higher Education, The University of Melbourne and Griffith Institute for Higher Education, Griffith University.
Johnson, T. F., & Butts, D. P. (1983). The relationship among college science student achievement, engaged time, and personal characteristics. Journal of Research in Science Teaching, 20(4), 357-366.
Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: The Guilford Press.
Krause, K.-L. (2005). Engaged, inert or otherwise occupied? Deconstructing the 21st century undergraduate student. In James Cook University Symposium, Sharing Scholarship in Learning and Teaching: Engaging Students. Townsville/Cairns, Queensland.
Krause, K.-L., & Coates, H. (2008). Students’ engagement in first-year university. Assessment & Evaluation in Higher Education, 33(5), 493-505.
Kuh, G. D. (2002). The National Survey of Student Engagement: Conceptual framework and overview of psychometric properties. In. Bloomington, IN: Indiana University Center for Postsecondary Research.
Kuh, G. D., Gonyea, R. M., & Palmer, M. (2001). The disengaged commuter student: Fact or fiction? Commuter Perspectives, 27(1), 2-5.
Langan, A. M., Wheater, C. P., Shaw, E. M., Haines, B. J., Cullen, W. R., Boyle, J. C., Penney, D., Oldekop, J. A., Ashcroft, C., & Lockey, L. (2005). Peer assessment of oral presentations: effects of student gender, university affiliation and participation in the development of assessment criteria. Assessment & Evaluation in Higher Education, 30(1), 21-34.
Lejk, M., Wyvill, M., & Farrow, S. (1996). A survey of methods of deriving individual grades from group assessments. Assessment & Evaluation in Higher Education, 21(3), 267-280.
Levi, D., & Cadiz, D. (1998). Evaluating team work on student projects: The use of behaviorally anchored scales to evaluate student performance. In. Cal Poly, San Luis Obispo, CA: Psychology and Human Development Department, US Department of Education.
Lingard, R. W. (2010). Teaching and assessing teamwork skills in engineering and computer science. Journal of Systemics, Cybernetics and Informatics, 8(1), 34-37.
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: the problem of capitalization on chance. Psychological bulletin, 111(3), 490-504.
Macdonald, J. (2004). Developing competent e-learners: the role of assessment. In Assessment & Evaluation in Higher Education (Vol. 29, pp. 215-226): Carfax Publishing Company.
Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal, 37(1), 153-184.
McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological methods, 7(1), 64-82.
Mishra, A., Catchpole, K., & McCulloch, P. (2009). The Oxford NOTECHS System: Reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Quality and Safety in Health Care, 18(2), 104-108.
Morgan, B. B., Glickman, A. S., Woodward, E. A., Blaiwes, A. S., & Salas, E. (1986). Measurement of team behaviors in a Navy environment. In. Orlando, FL: Naval Training Systems Center, Human Factors Division.
National Survey of Student Engagement. (2006). Engaged learning: Fostering success for all students In. Bloomington, IN: Indiana University Center for Postsecondary Research, School of Education.
Newmann, F. M. (1992). Student engagement and achievement in American Secondary Schools. In. New York, NY: Teachers College, Columbia University.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill, Inc.
Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and methodologies. Computers & Education, 57(1), 1098-1108.
Oslin, J. R. (2010). Identifying programmer ability using peer evaluation: An empirical investigation. Unpublished M.Sc. thesis, The University of Alabama Tuscaloosa, Alabama.
Perna, L. W., Ruby, A., Boruch, R. F., Wang, N., Scull, J., Ahmad, S., & Evans, C. (2014). Moving through MOOCs: Understanding the progression of users in Massive Open Online Courses. Educational Researcher, 43(9), 421-432, doi: 10.3102/0013189X14562423.
Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12(4), 531-544.
Porter, A. C., & Brophy, J. (1988). Synthesis of research on good teaching: Insights from the work of the institute for research on teaching. Educational Leadership, 45(8), 74-85.
Rafferty, A., Ball, J., & Aiken, L. (2001). Are teamwork and professional autonomy compatible, and do they result in improved hospital care? Quality in Health Care, 10(suppl 2), ii32-ii37.
Roadrangka, V., & Yeany, R. H. (1985). A study of the relationship among type and quality of implementation of science teaching strategy, student formal reasoning ability, and student engagement. Journal of Research in Science Teaching, 22(8), 743-760.
Sams, D. (2005). An empirical examination of job stress and management of emotionally-based behavior: Frontline social service personnel perspective. Unpublished Ph.D. dissertation, University of South Florida.
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338.
Segars, A. H. (1997). Assessing the unidimensionality of measurement: a paradigm and illustration within the context of information systems research. Omega International Journal of Management Science, 25(1), 107-121.
Sezer, A., & Tokcan, H. (2003). İşbirliğine dayalı öğrenmenin coğrafya dersinde akademik başarı üzerine etkisi. Gazi Eğitim Fakültesi Dergisi, 23(3), 227-242.
Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18(2), 158-176.
Spiller, D. (2009). Assessment matters: Self-assessment and peer assessment. Teaching Development Unit, The University of Waikato: Hamilton, New Zealand.
Stevens, J. (1996). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Publishers.
Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52(2), 302-312.
Tinio, M. F. O. (2009). Academic engagement scale for grade school students. Assessment Handbook, 2, 64-75.
Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276.
Topping, K. J., Smith, E. F., Swanson, I., & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment & Evaluation in Higher Education, 25(2), 149-169.
Trowler, V. (2010). Student engagement literature review. York: Higher Education Academy.
Usluel, Y. K., Aşkar, P., & Baş, T. (2008). A structural equation model for ICT usage in higher education. Educational Technology & Society, 11(2), 262-273.
Wang, L., MacCann, C., Zhuang, X., Liu, O. L., & Roberts, R. D. (2009). Assessing teamwork and collaboration in high school students: A multimethod approach. Canadian Journal of School Psychology, 24(2), 108-124.
Weller, J., Frengley, R., Torrie, J., Shulruf, B., Jolly, B., Hopley, L., Hendersdon, K., Dzendrowskyj, P., Yee, B., & Paul, A. (2011). Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Quality & Safety, 20(3), 216-222.
Willey, K., & Gardner, A. (2009). Self and peer assessment: A necessary ingredient in developing and tracking students’ graduate attributes. Paper presented at the Proceedings of the Research in Engineering Education Symposium 2009, Palm Cove, QLD.
Willey, K., & Gardner, A. (2010). Investigating the capacity of self and peer assessment activities to engage students and promote learning. European Journal of Engineering Education, 35(4), 429-443.

How to cite this article

APA

Öncü, S. (2015). Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education. Eurasia Journal of Mathematics, Science and Technology Education, 11(3), 535-549. https://doi.org/10.12973/eurasia.2015.1343a

Vancouver

Öncü S. Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education. EURASIA J Math Sci Tech Ed. 2015;11(3):535-49. https://doi.org/10.12973/eurasia.2015.1343a

AMA

Öncü S. Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education. EURASIA J Math Sci Tech Ed. 2015;11(3), 535-549. https://doi.org/10.12973/eurasia.2015.1343a

Chicago

Öncü, Semiral. "Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education". Eurasia Journal of Mathematics, Science and Technology Education 2015 11 no. 3 (2015): 535-549. https://doi.org/10.12973/eurasia.2015.1343a

Harvard

Öncü, S. (2015). Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education. Eurasia Journal of Mathematics, Science and Technology Education, 11(3), pp. 535-549. https://doi.org/10.12973/eurasia.2015.1343a

MLA

Öncü, Semiral "Online Peer Evaluation for Assessing Perceived Academic Engagement in Higher Education". Eurasia Journal of Mathematics, Science and Technology Education, vol. 11, no. 3, 2015, pp. 535-549. https://doi.org/10.12973/eurasia.2015.1343a