Measuring Student Character in a Fair and Contextual Manner: Development of Local Culture-Based Instruments with the IRT Model

Vinsensius Bawa Toron(1), Marzuki Marzuki(2), Ali Muhson(3), Florens Maxi Un Bria(4), Hermania Bhoki(5), Fidelis Regi Waton(6),

DOI: https://doi.org/10.46843/jiecr.v7i2.2744


(1) Program Studi Penelitian dan Evaluasi Pendidikan (PEP) Universitas Negeri Yogyakarta, Indonesia; Program Studi Pendidikan Keagamaan Katolik STP Reinha Larantuka, Indonesia
(2) Program Studi Penelitian dan Evaluasi Pendidikan (PEP) Universitas Negeri Yogyakarta, Indonesia
(3) Program Studi Penelitian dan Evaluasi Pendidikan (PEP) Universitas Negeri Yogyakarta, Indonesia
(4) Program Studi Pendidikan Keagamaan Katolik STIPAS Kupang, Indonesia
(5) Program Studi Pendidikan Keagamaan Katolik STP Reinha Larantuka, Indonesia
(6) Kölner Hochschule für Katholische Theologie, Germany

Abstract

The importance of character education in the formation of a generation that is intellectually endowed but also morally and socially accountable should not be underrated. Nonetheless, evaluating student character remains a problem, especially across diverse local cultural backgrounds. In this research, an attempt was made to come up with a measure of character among junior high school students based on the local cultural values, and to achieve this purpose, the current psychometric theory of the item Response Theory (IRT) was used in the development of the instrument, as well as expert and data validation through interviews, questionnaires, and observations. The Generalised Partial Credit Model (GPCM) was the most suitable as determined in the analysis, and all the 68 items were highly conforming. The unidimensionality assumption was met (MSA = 0.977; first-factor variance = 77.57%), and the parameter correlations implied invariance and fairness across student groups. The instrument has a total information value of 403.08, a standard error of 0.05, and an information area probability of 98.52%, which indicates high precision and coverage of the measurement. Qualitative findings proved cultural relevance and semantic clarity. The study will contribute to the field of educational evaluation by offering a valid, reliable, and culture-adaptive tool that can serve as a guide for both research and research-practice in character education across diverse settings.

Keywords

assessment; item response model (irm); psychometrics; student personality; school culture

Full Text:

PDF

References

Adu, J., Owusu, M. F., Martin-Yeboah, E., Gavidia, L. A. P., & Gyamfi, S. (2022). A Discussion of Some Controversies in Mixed Methods Research for Emerging Researchers. Methodological Innovations, 15(3), 321–330.

Ahmed, I., & Ishtiaq, S. (2021). Reliability and Validity: Importance in Medical Research. JPMA. The Journal of the Pakistan Medical Association, 71(10), 2401–2406.

Ambuehl, B., & Inauen, J. (2022). Contextualized Measurement Scale Adaptation: A 4-Step Tutorial for Health Psychology Research. International Journal of Environmental Research and Public Health, 19(19), 12775.

Anif, S., Prayitno, H. J., Narimo, S., Fuadi, D., Sari, D. P., & Adnan, M. (2021). Metacognition of Junior High School Students in Mathematics Problem Solving Based on Cognitive Style. Asian Journal of University Education, 17(1), 134–144.

Asmara, O. J., Morawska, A., Hoang, A., & Riany, Y. E. (2025). How Far Does the Apple Fall From the Tree? A Cross-Sectional Study of the Relationship Between Child and Parent Self-Regulation Mediated by Parenting Practices. International Journal of Behavioral Development, 01650254251327364.

Bader, M., Horsten, L. K., Hilbig, B. E., Zettler, I., & Moshagen, M. (2022). Measuring the Dark Core of Personality in German: Psychometric Properties, Measurement Invariance, Predictive Validity, and Self-Other Agreement. Journal of Personality Assessment, 104(5), 660–673.

Bazeley, P. (2010). Book Review: V. L. Plano Clark and J. W. Creswell (Eds.) The Mixed Methods Reader. Journal of Mixed Methods Research, 4(1), 79–81.

Biswas, A., & Verma, R. K. (2022). Engine of Entrepreneurial Intentions: Revisiting Personality Traits With Entrepreneurial Education. Benchmarking: An International Journal, 29(6), 2019–2044.

Bürkner, P. C. (2021). Bayesian Item Response Modeling in R With brms and Stan. Journal of Statistical Software, 100(5).

Caycho-Rodríguez, T., Vilca, L. W., Carbajal-León, C., White, M., Vivanco-Vidal, A., Saroli-Araníbar, D., Peña-Calero, B. N., & Moreta-Herrera, R. (2022). Coronavirus Anxiety Scale: New Psychometric Evidence for the Spanish Version Based on CFA and IRT Models in a Peruvian Sample. Death Studies, 46(5), 1090–1099.

Creswell, J. W. (2004). Designing a Mixed Methods Study in Primary Care. The Annals of Family Medicine, 2(1), 7–12.

Cruchinho, P., López-Franco, M. D., Capelas, M. L., Almeida, S., Bennett, P. M., Miranda Da Silva, M., Teixeira, G., Nunes, E., Lucas, P., & Gaspar, F. (2024). Translation, Cross-Cultural Adaptation, and Validation of Measurement Instruments: A Practical Guideline for Novice Researchers. Journal of Multidisciplinary Healthcare, 17, 2701–2728.

Culp, M. E., & Salvador, K. (2021). Music Teacher Education Program Practices: Preparing Teachers to Work With Diverse Learners. Journal of Music Teacher Education, 30(2), 51–64.

Dai, S., Vo, T. T., Kehinde, O. J., He, H., Xue, Y., Demir, C., & Wang, X. (2021). Performance of Polytomous IRT Models With Rating Scale Data: An Investigation Over Sample Size, Instrument Length, and Missing Data. Frontiers in Education, 6, 721963.

Dawson, J., Laccos-Barrett, K., Hammond, C., & Rumbold, A. (2022). Reflexive Practice as an Approach to Improve Healthcare Delivery for Indigenous Peoples: A Systematic Critical Synthesis and Exploration of the Cultural Safety Education Literature. International Journal of Environmental Research and Public Health, 19(11), 6691.

de Barros Ahrens, R., da Silva Lirani, L., & de Francisco, A. C. (2020). Construct Validity and Reliability of the Work Environment Assessment Instrument WE-10. International Journal of Environmental Research and Public Health, 17(20), 7364.

Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6(2), 80–88.

Dimyati, K., Nashir, H., Elviandri, E., Absori, A., Wardiono, K., & Budiono, A. (2021). Indonesia as a Legal Welfare State: A Prophetic-Transcendental Basis. Heliyon, 7(8), e07865.

Dodd, A. W. (1992). Educating for Character: How Our Schools Can Teach Respect and Responsibility. By Thomas Lickona. NASSP Bulletin, 76(545), 119–120.

Ekmekci, A., & Serrano, D. M. (2022). The Impact of Teacher Quality on Student Motivation, Achievement, and Persistence in Science and Mathematics. Education Sciences, 12(10), 649.

Fayers, P. (2004). Item Response Theory for Psychologists. Quality of Life Research, 13(3), 715–716.

Foronda, C., Prather, S., Baptiste, D. L., & Luctkar-Flude, M. (2022). Cultural Humility Toolkit. Nurse Educator, 47(5), 267–271.

Henninger, M., & Meiser, T. (2020). Different Approaches to Modeling Response Styles in Divide-by-Total Item Response Theory Models (Part 1): A Model Integration. Psychological Methods, 25(5), 560–576.

Holzinger, A., Lettner, S., Steiner-Hofbauer, V., & Melser, M. C. (2020). How to Assess? Perceptions and Preferences of Undergraduate Medical Students Concerning Traditional Assessment Methods. BMC Medical Education, 20(1), 312.

Hur, Y., & Seo, D. G. (2023). Development of a Character Qualities Test for Medical Students in Korea Using Polytomous Item Response Theory and Factor Analysis: A Preliminary Scale Development Study. Journal of Educational Evaluation for Health Professions, 20, 20.

Ibrahim, N. N., Naidu, N. B. M., Shegaram, P. C., & Ahmad, N. S. (2024). Character Education in the 21st Century: A Bibliometric Analysis on 2000-2024 Scopus Database. International Journal of Modern Education, 6(22), 741–757.

Islam, M. N., & Ahmed, D. H. (2021). Delaying the Temperature Fluctuations Through PCM Integrated Building Walls—Room Conditions, PCM Placement, and Temperature of the Heat Sources. Energy Storage, 3(5), e245.

Jurkowski, S., Ulrich, M., & Müller, B. (2023). Co-Teaching as a Resource for Inclusive Classes: Teachers' Perspectives on Conditions for Successful Collaboration. International Journal of Inclusive Education, 27(1), 54–71.

König, C. M., Karrenbauer, C., & Breitner, M. H. (2023). Critical Success Factors and Challenges for Individual Digital Study Assistants in Higher Education: A Mixed Methods Analysis. Education and Information Technologies, 28(4), 4475–4503.

Lacko, D., ?en?k, J., To?ík, J., Avsec, A., ?or?evi?, V., Genc, A., Haka, F., Šakoti?-Kurbalija, J., Mohori?, T., Neziri, I., & Suboti?, S. (2022). The Necessity of Testing Measurement Invariance in Cross-Cultural Research: Potential Bias in Cross-Cultural Comparisons With Individualism–Collectivism Self-Report Scales. Cross-Cultural Research, 56(2–3), 228–267.

Lang, J. W. B., & Tay, L. (2021). The Science and Practice of Item Response Theory in Organizations. Annual Review of Organizational Psychology and Organizational Behavior, 8(1), 311–338.

Lestari, L. (2021). Pembentukan Akhlakul Karimah dalam Buku Pendidikan Karakter Islam Karya Dr. Marzuki, M. Ag (Doctoral dissertation, Universitas Islam Negeri Sultan Syarif Kasim Riau).

Mardapi, D., & Kartowagiran, B. (2011). Pengembangan Instrumen Pengukur Hasil Belajar Nirbias dan Terskala Baku. Jurnal Penelitian dan Evaluasi Pendidikan, 15(2), 326–341.

Martin, C. S., Ramirez, C., Calvo, R., Muñoz-Martínez, Y., & Sharma, U. (2021). Chilean Teachers’ Attitudes Towards Inclusive Education, Intention, and Self-Efficacy to Implement Inclusive Practices. Sustainability, 13(4), 1–17.

McMillan, J. H., & Schumacher, S. (2010). Research in Education: Evidence-Based Inquiry, MyEducationLab Series. Pearson.

McNeish, D., & Wolf, M. G. (2023). Dynamic Fit Index Cutoffs for Confirmatory Factor Analysis Models. Psychological Methods, 28(1), 61–88.

Nagy, G., & Ulitzsch, E. (2022). A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models. Educational and Psychological Measurement, 82(5), 845–879.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847.

Omodan, B. I., Mpiti, P. T., & Mtsi, N. (2023). A Conceptual Analysis of What It Means to Decolonize the Curriculum. International Journal of Learning, Teaching and Educational Research, 22(7), 344–355.

Paju, B., Kajamaa, A., Pirttimaa, R., & Kontu, E. (2022). Collaboration for Inclusive Practices: Teaching Staff Perspectives From Finland. Scandinavian Journal of Educational Research, 66(3), 427–440.

Parker, L. J., Marx, K. A., Nkimbeng, M., Johnson, E., Koeuth, S., Gaugler, J. E., & Gitlin, L. N. (2023). It's More Than Language: Cultural Adaptation of a Proven Dementia Care Intervention for Hispanic/Latino Caregivers. The Gerontologist, 63(3), 558–567.

Patton, M. Q. (2002). Two Decades of Developments in Qualitative Inquiry: A Personal, Experiential Perspective. Qualitative Social Work, 1(3), 261–283.

Powell, T., Wegmann, K. M., & Backode, E. (2021). Coping and Post-Traumatic Stress in Children and Adolescents After an Acute Onset Disaster: A Systematic Review. International Journal of Environmental Research and Public Health, 18(9), 4865.

Pratama, D. (2020). Analisis Kualitas Tes Buatan Guru Melalui Pendekatan Item Response Theory (IRT) Model Rasch. Tarbawy: Jurnal Pendidikan Islam, 7(1), 61–70.

Rahmawati, Y., Taylor, E., Taylor, P. C., Ridwan, A., & Mardiah, A. (2022). Students' Engagement in Education as Sustainability: Implementing an Ethical Dilemma-STEAM Teaching Model in Chemistry Learning. Sustainability, 14(6), 3554.

Ribeiro, D. L., Costa, M., Helmich, E., Jaarsma, D., & De Carvalho?Filho, M. A. (2021). 'I Found Myself a Despicable Being!': Medical Students Face Disturbing Moral Dilemmas. Medical Education, 55(7), 857–871.

Sharon E., C., Lauren J., J., Jordan R., C., Abby A., K., Sharon K., P., Melody, R., Michele, K.-F., Negar, G., & Regine Beliard, V. (2022). A Systematic Review of Global Health Assessment for Education in Healthcare Professions. Annals of Global Health, 88(1), 1.

Shim, H., Bonifay, W., & Wiedermann, W. (2022). Parsimonious Asymmetric Item Response Theory Modeling With the Complementary Log-Log Link. Behavior Research Methods, 55(1), 200–219.

Shiyanbola, O. O., Rao, D., Bolt, D., Brown, C., Zhang, M., & Ward, E. (2021). Using an Exploratory Sequential Mixed Methods Design to Adapt an Illness Perception Questionnaire for African Americans With Diabetes: The Mixed Data Integration Process. Health Psychology and Behavioral Medicine, 9(1), 796–817.

Sinval, J., Vazquez, A. C. S., Hutz, C. S., Schaufeli, W. B., & Silva, S. (2022). Burnout Assessment Tool (BAT): Validity Evidence From Brazil and Portugal. International Journal of Environmental Research and Public Health, 19(3), 1344.

Souissi, S., Chamari, K., & Bellaj, T. (2022). Assessment of Executive Functions in School-Aged Children: A Narrative Review. Frontiers in Psychology, 13, 991699.

St?nculescu, E. (2023). The Bergen Social Media Addiction Scale Validity in a Romanian Sample Using Item Response Theory and Network Analysis. International Journal of Mental Health and Addiction, 21(4), 2475–2492.

Stephenson, C. R., Bonnes, S. L., Sawatsky, A. P., Richards, L. W., Schleck, C. D., Mandrekar, J. N., Beckman, T. J., & Wittich, C. M. (2020). The Relationship Between Learner Engagement and Teaching Effectiveness: A Novel Assessment of Student Engagement in Continuing Medical Education. BMC Medical Education, 20(1), 403.

Suprapto, N., Prahani, B. K., & Cheng, T. H. (2021). Indonesian Curriculum Reform in Policy and Local Wisdom: Perspectives From Science Education. Jurnal Pendidikan IPA Indonesia, 10(1), 69–80.

Syarnubi, S., Mansir, F., Purnomo, M. E., Harto, K., & Hawi, A. (2021). Implementing Character Education in Madrasah. Jurnal Pendidikan Islam, 7(1), 77–94.

Toraman, Ç., Karada?, E., & Polat, M. (2022). Validity and Reliability Evidence for the Scale of Distance Education Satisfaction of Medical Students Based on Item Response Theory (IRT). BMC Medical Education, 22(1), 94.

Tzima, S., Styliaras, G., & Bassounas, A. (2020). Revealing Hidden Local Cultural Heritage Through a Serious Escape Game in Outdoor Settings. Information, 12(1), 10.

Uher, J. (2021). Psychometrics Is Not Measurement: Unraveling a Fundamental Misconception in Quantitative Psychology and the Complex Network of Its Underlying Fallacies. Journal of Theoretical and Philosophical Psychology, 41(1), 58–84.

Veckalne, R., & Tambovceva, T. (2022). The Role of Digital Transformation in Education in Promoting Sustainable Development. Virtual Economics, 5(4), 65–86.

Wang, D., Zhu, F., Xi, S., Niu, L., Tebes, J. K., Xiao, S., & Yu, Y. (2021). Psychometric Properties of the Multidimensional Scale of Perceived Social Support (MSPSS) Among Family Caregivers of People With Schizophrenia in China. Psychology Research and Behavior Management, 14, 1201–1209.

Wraae, B., Brush, C., & Nikou, S. (2022). The Entrepreneurship Educator: Understanding Role Identity. Entrepreneurship Education and Pedagogy, 5(1), 3–35.

Wright, M., Getta, A., Green, A., Kickett, U., Kickett, A., McNamara, A., McNamara, U., Newman, A., Pell, A., Penny, A., Wilkes, U., Wilkes, A., Culbong, T., Taylor, K., Brown, A., Dudgeon, P., Pearson, G., Allsop, S., Lin, A., … O'Connell, M. (2021). Co-Designing Health Service Evaluation Tools That Foreground First Nation Worldviews for Better Mental Health and Wellbeing Outcomes. International Journal of Environmental Research and Public Health, 18(16), 8555.

Xie, Y., & DeVellis, R. F. (1992). Scale Development: Theory and Applications. Contemporary Sociology, 21(6), 876.

Yang, Y., Li, G., Su, Z., & Yuan, Y. (2021). Teacher's Emotional Support and Math Performance: The Chain Mediating Effect of Academic Self-Efficacy and Math Behavioral Engagement. Frontiers in Psychology, 12, 651608.

Zieli?ska, A., Lebuda, I., & Karwowski, M. (2022). Scaling the Creative Self: An Item Response Theory Analysis of the Short Scale of Creative Self. Creativity Research Journal, 34(4), 431–444.

Refbacks

  • There are currently no refbacks.


View My Stats