Design and content validity of mathematics creative problem-solving ability instrument for junior high school students

Himmatul Ulya; Sugiman; Raden Rosnawati

doi:10.29333/ejmste/14661

Full Text (PDF)

Himmatul Ulya ¹ ² ^* , Sugiman ¹ , Raden Rosnawati ¹

More Detail

¹ Universitas Negeri Yogyakarta, Sleman, INDONESIA² Universitas Muria Kudus, Kudus, INDONESIA^* Corresponding Author

Abstract

This research aims to design and validate the contents of the mathematical creative problem-solving (MCPS) instrument to measure creative problem-solving skills. This instrument consists of four aspects, each with related items representing latent variables for promoting students’ MCPS. An instrument construct that contains aspects of divergent and convergent thinking is presented. The stimulus provided is real problems in everyday life (real-world situations) with tiered levels of structure of the observed learning outcome (SOLO) taxonomy. The question structure developed is in the form of near-transfer and far-transfer questions. The content validity process is presented by seven expert validators experienced in mathematics, mathematics education, and measurement. The level of agreement between expert assessments is determined using the Aiken formula. Aiken coefficient for all items is above the good threshold. The instrument developed has demonstrated strong content validity and is recommended for measuring MCPS skills of junior high school students.

Keywords

SOLO taxonomy
design & content validity
MPCS instrument
near transfer & far transfer

License

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Article Type: Research Article

EURASIA J Math Sci Tech Ed, Volume 20, Issue 6, June 2024, Article No: em2462

https://doi.org/10.29333/ejmste/14661

Publication date: 01 Jun 2024

Online publication date: 30 May 2024

Article Views: 356

Article Downloads: 144

Open Access References How to cite this article

References

Aiken, L. R. (1985). Three coefficients to analyze the reliability and validity of rating. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012
Aktas, M. C., & Tabak, S. (2018). Turkish adaptation of math and me survey: A validity and reliability study. European Journal of Educational Research, 7(3), 707-714. https://doi.org/10.12973/eu-jer.7.3.707
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Brooks/Cole Publishing Company.
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214-221. https://doi.org/10.1016/j.sapharm.2018.03.066
Amalina, I. K., & Vidákovich, T. (2022). An integrated STEM-based mathematical problem-solving test: Developing and reporting psychometric evidence. Journal on Mathematics Education, 13(4), 587-604. https://doi.org/10.22342/jme.v13i4.pp587-604
Amalina, I. K., & Vidákovich, T. (2023). Assessment of domain-specific prior knowledge: A development and validation of mathematical problem-solving test. International Journal of Evaluation and Research in Education, 12(1), 468-476. https://doi.org/10.11591/ijere.v12i1.23831
Arp, R. (2008). Scenario visualization: An evolutionary account of creative problem solving. American Journal of Human Biology, 21(1). https://doi.org/10.1002/ajhb.20848
Arthur, C., Badertscher, E., Goldenberg, E. P., Moeller, B., McLeod, M., Nikula, J., & Reed, K. (2017). Strategies to improve all students’ mathematics learning and achievement. Education Development Center, Inc.
Authié, C. N., Poujade, M., Talebi, A., Defer, A., Zenouda, A., Coen, C., Mohand-Said, S., Chaumet-Riffaud, P., Audo, I., & Sahel, J.-A. (2023). Development and validation of a novel mobility test for rod-cone dystrophies, from reality to virtual reality. American Journal of Ophthalmology, 258, 43-54. https://doi.org/10.1016/j.ajo.2023.06.028
Bicer, A., Lee, Y., Capraro, R. M., Capraro, M. M., Barroso, L. R., & Rugh, M. (2019). Examining the effects of STEM PBL on students’ divergent thinking attitudes related to creative problem solving. In Proceedings of the 2019 IEEE Frontiers in Education Conference (pp. 1-6). IEEE. https://doi.org/10.1109/FIE43999.2019.9028431
Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed learning outcome). Academic Press. https://doi.org/10.1177/000494418302700311
Brophy, D. R. (2001). Comparing the attributes, activities, and performance of divergent, convergent, and combination thinkers. Creativity Research Journal, 13(3-4), 439-455. https://doi.org/10.1207/S15326934CRJ1334_20
Buitrago, R., Salinas, J., & Boude, O. (2023). Validation of a model for the formalization of personal learning pathways through expert judgment. Journal of Higher Education Theory and Practice, 23(12), 224-241. https://doi.org/10.33423/jhetp.v23i12.6283
Byrnes, J. P. (1996). Cognitive development and learning in instructional contexts. Allyn and Bacon.
Chamberlin, S. A., Liljedahl, P., & Savić, M. (2022). Mathematical creativity: A developmental perspective. Springer. https://doi.org/10.1007/978-3-031-14474-5
Chen, Y. W., Chang, W. H., & Kuo, C. C. (2016). A comparative study of the divergent problem solving abilities of mathematically and scientifically talented students and nongifted students. Thinking Skills and Creativity, 22, 247-255. https://doi.org/10.1016/j.tsc.2016.10.009
Chirove, M., Mogari, D., & Ugorji, O. (2022). Students’ mathematics-related belief systems and their strategies for solving non-routine mathematical problems. Waikato Journal of Education, 27(3), 101-121. https://doi.org/10.15663/wje.v27i3.822
Cho, S., & Hwang, D.-J. (2007). Math creative problem solving ability test for identification of the mathematically gifted middle school students. Journal of Gifted/Talented Education, 17(1), 1-26. https://doi.org/10.9722/JGTE.2021.31.1.1
Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry & research design: Choosing among five approaches. SAGE. https://doi.org/10.13187/rjs.2017.1.30
Cukurova, M., Bennett, J., & Abrahams, I. (2018). Students’ knowledge acquisition and ability to apply knowledge into different science contexts in two different independent learning settings. Research in Science and Technological Education, 36(1), 17-34. https://doi.org/10.1080/02635143.2017.1336709
Daniels, J. S., Moreau, D., & Macnamara, B. N. (2022). Learning and transfer in problem solving progressions. Journal of Intelligence, 10(4), 85. https://doi.org/10.3390/jintelligence10040085
de Vink, I. C., Willemsen, R. H., Lazonder, A. W., & Kroesbergen, E. H. (2022). Creativity in mathematics performance: The role of divergent and convergent thinking. British Journal of Educational Psychology, 92(2), 484-501. https://doi.org/10.1111/bjep.12459
del Pozo-Herce, P., Martínez-Sabater, A., Chover-Sierra, E., Gea-Caballero, V., Satústegui-Dordá, P. J., Saus-Ortega, C., Tejada-Garrido, C. I., Sánchez-Barba, M., Pérez, J., Juárez-Vela, R., Santolalla-Arnedo, I., & Baca-García, E. (2023). Application of the Delphi method for content validity analysis of a questionnaire to determine the risk factors of the Chemsex. Healthcare, 11(21), 2905. https://doi.org/10.3390/healthcare11212905
Erdem, C. (2020). Introduction to 21^st century skills and education. In 21^st century skills and education. Cambridge Scholars Publishing.
Espinoza Salfate, L., Guerrero, G., Barbé Farré, J., & Márquez Salinas, F. (2023). Design and validation of a classroom observation instrument to evaluate the quality of mathematical activity from a gender perspective. Education Sciences, 13(3), 266. https://doi.org/10.3390/educsci13030266
Fernandes, S., Abelha, M., & Albuquerque, A. S. (2022). The student journey in PBL: Using individual portfolios to promote self-reflection and assessment as learning. International Symposium on Project Approaches in Engineering Education, 12, 207-211. https://doi.org/10.5281/zenodo.7058095
Gavaz, H. O., Yazgan, Y., & Arslan, C. (2021). Non-routine problem solving and strategy flexibility: A quasi-experimental study. Journal of Pedagogical Research, 5(3), 40-54. https://doi.org/10.33902/jpr.2021370581
Ghazali, N., Sahari Nordin, M., Hashim, S., & Hussein, S. (2018). Measuring content validity: Students self-efficacy and meaningful learning in massive open online course (MOOC) scale. In Proceedings of the 3^rd International Conferences on Education in Muslim Society (pp. 128-133). https://doi.org/10.2991/icems-17.2018.25
Ghorbanpour, E., Abbasian, G. R., & Mohseni, A. (2021). Assessment alternatives in developing L2 listening ability: Assessment FOR, OF, AS learning or integration? Assessment x approach. International Journal of Language Testing, 11(1), 36-57.
Gilligan, K. A., Thomas, M. S. C., & Farran, E. K. (2020). First demonstration of effective spatial training for near transfer to spatial performance and far transfer to a range of mathematics skills at 8 years. Developmental Science, 23(4), e12909. https://doi.org/10.1111/desc.12909
Gizzi, E., Nair, L., Chernova, S., & Sinapov, J. (2022). Creative problem solving in artificially intelligent agents: A survey and framework. International Joint Conference on Artificial Intelligence, 75(2022), 857-911. https://doi.org/10.24963/ijcai.2023/777
Hook, P. (2014). About SOLO taxonomy. https://leadinglearnerdotme.files.wordpress.com/2014/09/about-solo-taxonomy-by-pam-hook-pdf.pdf
Hrnjičić, A., Alihodžić, A., Čunjalo, F., & Hamzić, D. K. (2022). Development of an item bank for measuring students’ conceptual understanding of real functions. European Journal of Science and Mathematics Education, 10(4), 455-470. https://doi.org/10.30935/scimath/12222
Incebacak, B., & Ersoy, E. (2018). Ortaokul öğrencilerinin yaratıcı problem çözme becerileri [Creative problem solving skills of middle school students]. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi [Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education], 12(1), 1-24. https://doi.org/10.17522/balikesirnef.437352
Isrok’atun. (2014). Situation-based learning untuk meningkatkan kemampuan creative problem solving matematis siswa [Situation-based learning to improve students’ creative mathematical problem solving abilities]. Universitas Pendidikan Indonesia.
Jaiswal, P. (2019). Using constructive alignment to foster teaching learning processes. English Language Teaching, 12(6), 10. https://doi.org/10.5539/elt.v12n6p10
Johnny, J., Abdullah, A. H., Abu, M. S., Mokhtar, M., & Atan, N. A. (2017). Difficulties in reasoning among high achievers when doing problem solving in mathematics. Man in India, 97(12), 61-70.
Kahveci, M., Kahveci, A., Mansour, N., & Alarfaj, M. M. (2016). Construct validity and reliability measures of scores from the science teachers’ pedagogical discontentment (STPD) scale. EURASIA Journal of Mathematics, Science and Technology Education, 12(3), 549-558. https://doi.org/10.12973/eurasia.2015.1417a
Karamustafaoğlu, O., & Pektaş, H. M. (2023). Developing students’ creative problem solving skills with inquiry-based STEM activity in an out-of-school learning environment. Education and Information Technologies, 28, 7651-7669. https://doi.org/10.1007/s10639-022-11496-5
Kasemsap, K. (2017). Advocating problem-based learning and creative problem-solving skills in global education. In C. Zhou (Ed.), Handbook of research on creative problem-solving skill development in higher education (pp. 351-377). IGI Global. https://doi.org/10.4018/978-1-5225-0643-0.ch016
Keles, T. (2022). Investigation of high school students’ creative problem-solving attributes. Journal of Pedagogical Research, 6(4), 66-83. https://doi.org/10.33902/JPR.202215433
Kim, H., Cho, S., & Ahn, D. (2003). Development of mathematical creative problem solving ability test for identification of the gifted in math. Gifted Education International, 18, 164-174. https://doi.org/10.1177/026142940301800206
Klymchuk, S., Thomas, M., Alexander, J., Evans, T., Gulyaev, S., Liu, W., Novak, J., Murphy, P., Stephens, J., & Zaliwski, A. (2020). Investigating the impact of non-routine problem-solving on creativity, engagement, and intuition of STEM tertiary students. Teaching and Learning Research Initiative. https://eric.ed.gov/?id=ED608044
Kozakli Ulger, T., & Yazgan, Y. (2021). Non-routine problem-posing skills of prospective mathematics teachers. Eurasian Journal of Educational Research, 21(94), 147-168. https://doi.org/10.14689/ejer.2021.94.7
Lee, K. H., Kim, Y., & Lim, W. (2020). Risks of aiming to kill two birds with one stone: The effect of mathematically gifted and talented students in the dual realities of special schooling. Mathematical Thinking and Learning, 23(4), 271-290. https://doi.org/10.1080/10986065.2020.1784696
Lee, K. S., Hwang, D., & Seo, J. J. (2003). A development of the test for mathematical creative problem solving ability. Journal of the Korea Society of Mathematical Education, Series D: Research in Mathematical Education, 7(3), 163-189.
Lee, M., & Cha, D. (2016). A comparison of generalizability theory and many facet Rasch measurement in an analysis of mathematics creative problem solving test. Journal of Curriculum and Evaluation, 19(2), 251-279. https://doi.org/10.29221/jce.2016.19.2.251
Leikin, R., Leikin, M., Paz-Baruch, N., Waisman, I., & Lev, M. (2017). On the four types of characteristics of super mathematically gifted students. High Ability Studies, 28(1), 107-125. https://doi.org/10.1080/13598139.2017.1305330
Leminger, E. A. (2020). Human resource professional’s attitudes towards creativity [Master’s project, State University of New York].
Lim, H. L., Wun, T. Y., & Idris, N. (2010). Superitem test: As an alternative assessment tool. International Journal for Mathematics Teaching and Learning.
Maker, C. J. (2020). Identifying exceptional talent in science, technology, engineering, and mathematics: Increasing diversity and assessing creative problem-solving. Journal of Advanced Academics, 31(3), 161-210. https://doi.org/10.1177/1932202X20918203
McMillan, J. H. (2018). Classroom assessment: Principles and practice that enhance student learning and motivation. Pearson.
Munahefi, D. N., Khoirunnisa, K., Dwijanto, Mulyono, Fariz, R., & Noverianto, B. (2021). Development of open ended project based learning model assisted by GeoGebra for mathematical creative thinking ability. In Proceedings of the 8^th International Conference on Mathematics, Science and Education. https://doi.org/10.1063/5.0126736
Novriania, M. R., & Surya, E. (2017). Analysis of student difficulties in mathematics problem solving ability at MTs SWASTA IRA Medan. International Journal of Sciences: Basic and Applied Research, 33(3), 63-75.
OECD. (2012). PISA 2012 creative problem solving. https://www.oecd.org/pisa/innovation/creative-problem-solving/
OECD. (2019). PISA 2021 creative thinking framework. OECD Publishing.
Oraklibel, R. D., Ulkebas, S. D., & Oygur, I. (2018). Creative problem-solving assessment and product design education. In Proceedings of the International Conference on Engineering and Product Design Education.
Orón, J. V., & Lizasoain, I. (2023). Achieving transfer from mathematics learning. Education Sciences, 13(2), 161. https://doi.org/10.3390/educsci13020161
Phonapichat, P., Wongwanich, S., & Sujiva, S. (2014). An analysis of elementary school students’ difficulties in mathematical problem solving. Procedia-Social and Behavioral Sciences, 116, 3169-3174. https://doi.org/10.1016/j.sbspro.2014.01.728
Proctor, T. (2005). Creative problem solving for managers–Developing skills for decision making and innovation. Routledge. https://doi.org/10.1177/1350507601321014
Rahayuningsih, S., Sirajuddin, S., & Ikram, M. (2021). Using open-ended problem-solving tests to identify students’ mathematical creative thinking ability. Participatory Educational Research, 8(3), 285-299. https://doi.org/10.17275/per.21.66.8.3
Rebello, N. S., Cui, L., Bennett, A. G., Zollman, D. A., & Ozimek, D. J. (2007). Transfer of learning in problem solving in the context of mathematics and physics. In D. H. Jonassen (Ed.), Learning to solve complex scientific problems. Routledge. https://doi.org/10.4324/9781315091938-10
Risnanosanti, Susyla, D., & Syofiana, M. (2019). Developing students critical thinking ability through lesson study. Journal of Physics: Conference Series, 1320, 012005. https://doi.org/10.1088/1742-6596/1320/1/012005
Roebianto, A., Savitri, S. I., Aulia, I., Suciyana, A., & Mubarokah, L. (2023). Content validity: Definition and procedure of content validation in psychological research. TPM-Testing, Psychometrics, Methodology in Applied Psychology, 30(1), 5-18. https://doi.org/10.4473/TPM30.1.1
Rosaini, R., Budiyono, B., & Pratiwi, H. (2019). Mathematics teacher supporting higher order thinking skill of students through assessment as learning in instructional model. Journal of Physics: Conference Series, 1157(3), 032076. https://doi.org/10.1088/1742-6596/1157/3/032076
Rotigel, J. V., & Fello, S. (2004). Mathematically gifted students: How can we meet their needs? Gifted Child Today, 27(4), 46-51. https://doi.org/10.4219/gct-2004-150
Rubenstein, L. D. V., Callan, G. L., Speirs Neumeister, K., Ridgley, L. M., & Hernández Finch, M. (2020). How problem identification strategies influence creativity outcomes. Contemporary Educational Psychology, 60, 101840. https://doi.org/10.1016/j.cedpsych.2020.101840
Ryan, V., Fitzmaurice, O., & O’donoghue, J. (2022). Student interest and engagement in mathematics after the first year of secondary education. European Journal of Science and Mathematics Education, 10(4), 436-454. https://doi.org/10.30935/SCIMATH/12180
Sala, G., Deniz Aksayli, N., Semir Tatlidil, K., Tatsumi, T., Gondo, Y., & Gobet, F. (2019). Near and far transfer in cognitive training: A second-order meta-analysis. Collabra: Psychology, 5(1), 18. https://doi.org/10.1525/collabra.203
Sipayung, T. N., Imelda, Siswono, T. Y. E., & Masriyah. (2021). The preliminary study of students’ creative problem-solving ability. Journal of Physics: Conference Series, 1836, 012052. https://doi.org/10.1088/1742-6596/1836/1/012052
Suherman, S., & Vidákovich, T. (2022). Assessment of mathematical creative thinking: A systematic review. Thinking Skills and Creativity, 44, 101019. https://doi.org/10.1016/j.tsc.2022.101019
Svensäter, G., & Rohlin, M. (2022). Assessment model blending formative and summative assessments using the SOLO taxonomy. European Journal of Dental Education, 27(1), 149-157. https://doi.org/10.1111/eje.12787
Tan, S., & Maker, C. J. (2020). Assessing creative problem solving ability in mathematics: The discover mathematics assessment. Gifted and Talented International, 35(1), 58-71. https://doi.org/10.1080/15332276.2020.1793702
Teseo, R. F. (2019). Analyses of attribute patterns of mathematical creative problem-solving ability in 6^th grade students. St. John’s University.
Ulya, H., Rahayu, R., & Riyono, A. (2019). Integration of products assessment in mind mapping learning to enhance mathematical communication. Journal of Physics: Conference Series, 1175, 012142. https://doi.org/10.1088/1742-6596/1175/1/012142
Ulya, H., Sugiman, Rosnawati, R., & Retnawati, H. (2024). Technology-based learning interventions on mathematical problem-solving: A meta-analysis of research in Indonesia. International Journal of Evaluation and Research in Education, 13(1), 292-301. https://doi.org/10.11591/ijere.v13i1.26380
van Hooijdonk, M., Mainhard, T., Kroesbergen, E. H., & van Tartwijk, J. (2020). Creative problem solving in primary education: Exploring the role of fact finding, problem finding, and solution finding across tasks. Thinking Skills and Creativity, 37), 100665. https://doi.org/10.1016/j.tsc.2020.100665
van Hooijdonk, M., Mainhard, T., Kroesbergen, E. H., & Van Tartwijk, J. (2022). Examining the assessment of creativity with generalizability theory: An analysis of creative problem solving assessment tasks. Thinking Skills and Creativity, 43, 100994. https://doi.org/10.1016/j.tsc.2021.100994
Yazgan-Sağ, G. (2022). Views on mathematical giftedness and characteristics of mathematically gifted students-the case of prospective primary mathematics teachers. Mathematics Teaching-Research Journal, 14(5), 128-140.
Yuliani, A., Kusumah, Y. S., & Sumarmo, U. (2019). Mathematical creative problem solving ability and self-efficacy: A survey with eight grade students. Journal of Physics: Conference Series, 1157(3), 032097. https://doi.org/10.1088/1742-6596/1157/3/032097
Zhang, Q., Zhang, Q., Zhang, X., & Gao, C. (2023). Effects of rule variant reasoning in far transfer problem solving. Acta Psychologica Sinica, 55(1), 117-128. https://doi.org/10.3724/SP.J.1041.2023.00117

How to cite this article

APA

Ulya, H., Sugiman, & Rosnawati, R. (2024). Design and content validity of mathematics creative problem-solving ability instrument for junior high school students. Eurasia Journal of Mathematics, Science and Technology Education, 20(6), em2462. https://doi.org/10.29333/ejmste/14661

Vancouver

Ulya H, Sugiman, Rosnawati R. Design and content validity of mathematics creative problem-solving ability instrument for junior high school students. EURASIA J Math Sci Tech Ed. 2024;20(6):em2462. https://doi.org/10.29333/ejmste/14661

AMA

Ulya H, Sugiman, Rosnawati R. Design and content validity of mathematics creative problem-solving ability instrument for junior high school students. EURASIA J Math Sci Tech Ed. 2024;20(6), em2462. https://doi.org/10.29333/ejmste/14661

Chicago

Ulya, Himmatul, Sugiman, and Raden Rosnawati. "Design and content validity of mathematics creative problem-solving ability instrument for junior high school students". Eurasia Journal of Mathematics, Science and Technology Education 2024 20 no. 6 (2024): em2462. https://doi.org/10.29333/ejmste/14661

Harvard

Ulya, H., Sugiman, and Rosnawati, R. (2024). Design and content validity of mathematics creative problem-solving ability instrument for junior high school students. Eurasia Journal of Mathematics, Science and Technology Education, 20(6), em2462. https://doi.org/10.29333/ejmste/14661

MLA

Ulya, Himmatul et al. "Design and content validity of mathematics creative problem-solving ability instrument for junior high school students". Eurasia Journal of Mathematics, Science and Technology Education, vol. 20, no. 6, 2024, em2462. https://doi.org/10.29333/ejmste/14661