Cognitive assessment development: item analysis with classical and modern theories

Desi Putriani; Riza Dhifa Alzabarjud Putra; Karlina Anjani; Yurizka Melia Sari

doi:10.30762/f_m.v8i1.2668

Authors

Desi Putriani Mathematics Education Department, Universitas Negeri Surabaya, East Java, Indonesia
Riza Dhifa Alzabarjud Putra Mathematics Education Department, Universitas Negeri Surabaya, East Java, Indonesia
Karlina Anjani Mathematics Education Department, Universitas Negeri Surabaya, East Java, Indonesia
Yurizka Melia Sari Mathematics Education Department, Universitas Negeri Surabaya, East Java, Indonesia

DOI:

https://doi.org/10.30762/f_m.v8i1.2668

Keywords:

Assessment Cognitive, Characterisation Instrument Test, Classical Theory, Modern Theory, Partial Credit Model

Abstract

Understanding geometric concepts, particularly angles in circles, is fundamental in junior high school mathematics and is crucial in enhancing students' higher-order thinking skills. Assessing students’ cognitive understanding requires valid and reliable instruments that align with curriculum standards and student abilities. This study aims to construct and characterise a cognitive test instrument on angle material in circles through item analysis using Classical Test Theory with Microsoft Excel and Modern Test Theory with RStudio. A mixed-methods approach combining qualitative and quantitative techniques was used to collect data through observation and item analysis. The test instrument consists of five items to measure students’ understanding and analytical abilities. Content validation using the Aiken formula yielded an Aiken index of 1 for each item, indicating all items were valid. Classical analysis showed that items 1 and 3 were difficult, while items 2, 4, and 5 were of medium difficulty. Using the Partial Credit Model (PCM), all items fell within the medium-hard difficulty range. Power differential analysis revealed that item 3 was poorly constructed and should be removed. The results demonstrate that the developed instrument is valid and can differentiate levels of student understanding. This test is expected to support efforts to improve mathematics learning and better prepare students for numeracy literacy components in the Minimum Competency Assessment (MCA). The instrument also provides a framework for developing similar tools in other areas of mathematics education to enhance the accuracy of student learning evaluations.

Downloads

Download data is not yet available.

References

Amelia, R. N., & Kriswantoro, K. (2017). Implementation of item response theory to analyse test items quality and students’ ability in chemistry. JKPK (Jurnal Kimia Dan Pendidikan Kimia), 2(1), 1-12. https://doi.org/10.20961/jkpk.v2i1.8512

Azis, A., Handayani, I. T., Ferniati, F., Anggriana, N., & Aisyah, A. (2023). Analysis of students’ cognitive difficulties based on the revised Bloom’s taxonomy in solving mathematics problems. Journal Focus Action of Research Mathematic (Factor M), 6(1), 117–138. https://doi.org/10.30762/factor_m.v6i1.1057

Cahyani, L. N., Shodiq, L. J., & Agustin, D. R. (2022). Kemampuan literasi matematika siswa dalam memecahkan soal TIMMS konten aljabar ditinjau dari pengetahuan metakognitif. Journal Focus Action of Research Mathematic (Factor M), 5(1), 31–51. https://doi.org/10.30762/f_m.v5i1.646

Fatimah, L. U., & Alfath, K. (2019). Analisis kesukaran soal, daya pembeda dan fungsi distraktor. Jurnal Komunikasi Dan Pendidikan Islam, 8(2), 47–64. https://doi.org/10.36668/jal.v8i2.115

Friyatni. (2018). Estimasi parameter tes dengan penskoran politomus menggunakan graded response model pada sampel kecil. JIPE (Jurnal Inovasi Pendidikan Ekonomi), 8(1), 22–31. https://doi.org/10.24036/01104490

Hasibuan, H. Y., Yuhana, Y., Santosa, C. A. H. F., Syamsuri, S., & Wahyudin, U. (2023). Menyelisik penelitian terkait diagnostik kognitif materi matematika di Indonesia melalui systematic literature review. Aksioma: Jurnal Program Studi Pendidikan Matematika, 12(2), 1762-1777. https://doi.org/10.24127/ajpm.v12i2.6886

Hewi, L., Shaleh, M., & IAIN Kendari, P. (2020). Refleksi hasil PISA (The Programme for International Student Assessment): Upaya perbaikan bertumpu pada pendidikan anak usia dini. Jurnal Golden Age, 4(1), 30–41. https://e-journal.hamzanwadi.ac.id/index.php/jga/article/view/2018

Hidayatullah, H., Safitri, R., & Suyanto, S. (2022). Model analisis butir instrumen tes biologi untuk penilaian akhir tahun menggunakan item response theory. Measurement in Educational Research (Meter), 2(1), 1-10. https://doi.org/10.33292/meter.v2i1.173

Jumini, S., Madnasri, S., Cahyono, E., & Parmin, P. (2023). Analisis kualitas butir soal pengukuran literasi sains melalui teori tes klasik dan Rasch model. Prosiding Seminar Nasional Pascasarjana, 6(1), 758-765 https://proceeding.unnes.ac.id/index.php/snpasca/article/view/2215

Kadek, N. R. P., & Sumandya, I. W. (2019). Penerapan partial credit model (PCM) dalam mengevaluasi tes uraian. Jurnal EMASAINS, 8(1), 77–85. https://doi.org/10.59672/emasains.v8i1.274

Kartikaningrum, D. M., & Muhtarom. (2024). Uji validitas dan reliabilitas bahan ajar menggunakan formula Aiken’s V dan SPSS.22 terhadap kemampuan berpikir kritis siswa. Pendas: Jurnal Ilmiah Pendidikan Dasar, 9(1), 879–885. https://journal.unpas.ac.id/index.php/pendas/article/view/11918

Laela, U. F., & Alfath, K. (2019). Analisis kesukaran soal, daya pembeda dan fungsi distraktor. Jurnal Komunikasi dan Pendidikan Islam, 8(2), 47–64. https://doi.org/10.36668/jal.v8i2.115

Masfufah, R., & Afriansyah, E. A. (2022). Analisis kemampuan literasi matematis ditinjau dari gaya belajar siswa SMP pada pembelajaran daring. Jurnal Perisai: Pendidikan dan Riset Ilmu Sains, 1(1), 1–13. https://doi.org/10.32672/perisai.v1i1.52

Mboeik, V. (2023). Literasi matematika siswa sekolah dasar. Jurnal Citra Pendidikan, 3(1), 781–788. https://doi.org/10.38048/jcp.v3i1.1421

Mega, A. M. P., & Madani, F. (2023). Analisisis assesmen autentik pembelajaran matematika sekolah dasar. Jurnal Elementaria Edukasia, 6(2), 778–788. https://doi.org/10.31949/jee.v6i2.5659

Nabil, N. R. A., Wulandari, I., Yamtinah, S., Ariani, S. R. D., & Ulfa, M. (2022). Analisis indeks Aiken untuk mengetahui validitas isi instrumen asesmen kompetensi minimum berbasis konteks sains kimia. Paedagogia, 25(2), 184-191. https://doi.org/10.20961/paedagogia.v25i2.64566

OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. OECD Publishing. https://www.oecd.org/en/publications/2013/02/pisa-2012-assessment-and-analytical-framework_g1g27388.html

OECD. (2023). PISA 2022 Results The State of Learning and Equity in Education (Volume I). OECD Publishing. https://doi.org/10.1787/53f23881-en

Ohiri, S. C., & Okoye, R. O. (2023). Application of classical test theory as linear modeling to test item development and analysis. International Research Journal of Modernization in Engineering Technology and Science, 5(10), 2152-2159. https://doi.org/10.56726/IRJMETS45379

Priyanto, A., Hartoyo, A., & Noviani, E. (2024). Pengembangan asesmen pembelajaran matematika untuk menguji kemampuan bernalar kritis dan kreatif siswa sekolah menengah atas. Journal of Education Research, 5(2), 1139–1146. https://doi.org/10.37985/jer.v5i2.975

Purnamatati, A. M. M., & Madani, F. (2023). Analisis asesmen autentik pembelajaran matematika sekolah dasar. Jurnal Elementaria Edukasia, 6(2), 778–788. https://doi.org/10.31949/jee.v6i2.5659

Rezigalla, A. A., Eleragi, A. M. E. S. A., Elhussein, A. B., Alfaifi, J., ALGhamdi, M. A., Al Ameer, A. Y., Yahia, A. I. O., Mohammed, O. A., & Adam, M. I. E. (2024). Item analysis: the impact of distractor efficiency on the difficulty index and discrimination power of multiple-choice items. BMC Medical Education, 24(1), 445-451. https://doi.org/10.1186/s12909-024-05433-y

Safaruddin, Anisa, & Saleh, M. A. (2012). Partial credit model (PCM) dalam penskoran politomi pada teori respon butir. Jurnal Matematika, Statistika Dan Komputasi, 9(1), 39–48. https://doi.org/10.20956/jmsk.v9i1.3397

Sarea, M. S., & Ruslan, R. (2019). Karakteristik butir soal: classical test theory vs item response theory. Didaktika: Jurnal Kependidikan, 13(1). https://doi.org/10.30863/didaktika.v13i1.296

Solichin, M. (2017). Analisis daya beda soal. Dirasat: Jurnal Manajemen Dan Pendidikan Islam, 2(2), 192–213. https://doi.org/10.26594/dirasat.v2i2.879

Sopandi, H., Supratman, & Dewi, S. V. (2024). The concept structure of student in mathematical literacy. Journal Focus Action of Research Mathematic (Factor M), 7(2), 102–114. https://doi.org/10.30762/f_m.v7i2.3757

Sumaryanta. (2021). Teori tes klasik & teori respon butir: Konsep & contoh penerapannya (R. O. Akbar, Ed.). CV. Confident.

Suciati, S., Munadi, S., & Sugiman. (2021). Estimation of test item parameters with polytomous item response using Partial Credit Model (PCM). In Proceedings of the 2nd International Conference on Innovation in Education and Pedagogy (ICIEP 2020), 608, 232-237. https://doi.org/10.2991/assehr.k.211219.042

Wibawa. (2019). Karakteristik butir soal tes ujian akhir semester hukum bisnis. Jurnal Pendidikan Akuntansi Indonesia, 17(1), 87–96. https://doi.org/10.21831/jpai.v17i1.26339

Yansa, H., & Retnawati, H. (2021). Identifikasi praktik dan hambatan guru dalam asesmen kognitif matematika di masa pandemi COVID-19. Jurnal Elemen, 7(1), 84–97. https://doi.org/10.29408/jel.v7i1.2585

Yustiandi, & Saepuzaman, D. (2024). Analisis karakteristik butir soal seleksi penerimaan mahasiswa baru berdasarkan pendekatan teori tes klasik. Jurnal Basicedu, 8(6), 4700–4706. https://doi.org/10.31004/basicedu.v8i6.9031

Zain, A. Z., Ramalis, T. R., & Muslim, M. (2022). Karakterisasi instrumen tes keterampilan berpikir kreatif berdasarkan analisis partial credit model. Jurnal Ilmiah Pendidikan Fisika, 6(1), 176-187. https://doi.org/10.20527/jipf.v6i1.4806