Item Analysis of Multiple-Choice Questions at the Department of Community Medicine, Wah Medical College, Pakistan
Psychometry-Item Analysis
Abstract
Objective: The objective of the study was to assess the quality of multiple-choice questions (MCQs) of three different assessments in the subject of Community Medicine by computing the difficulty index, discrimination index and reliability and to estimate the relationship between difficulty and discrimination indices.
Study Design: Retrospective observational study.
Place and Duration of Study: Department of Community Medicine at Wah Medical College from August to December 2018.
Materials and Methods: Three sets of MCQs were included in the study. Mean and standard deviation of difficulty and discrimination indices were calculated and one-way analysis of variance and Kruskal Wallis test were applied on difficulty and discrimination indices. The association was determined by Pearson correlation and considered significant at p value of < 0.05.
Results: The mean difficulty index of first term, second term and send-up examination were 41.5, 48.8 and 51.9 respectively. Mean discrimination indices were 0.28, 0.27 and 0.26 and reliability were 0.83, 0.81 and 0.79. In the study, 72% MCQs of the first term, 61.5 % of the second term and 63% of the send-up examinations were in the range 30-70% of difficulty. There was a significant difference in the difficulty indices of the three question papers. The correlation between discrimination and difficulty indices was curvilinear and positively correlated.
Conclusion: It is concluded that all three question papers have acceptable reliability, more than 65% MCQs have acceptable difficulty index and about 69% have good discriminatory power.
References
Islam Zu, Usmani A. Psychometric analysis of Anatomy MCQs in Modular examination. Pak J Med Sci. 2017; 33: 1138-43.
Gajjar S, Sharma R, Kumar P, Rana M. Item and test analysis to identify quality multiple choice questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian J Community Med. 2014; 39: 17-20.
Rahma NAA, Shamad MMA, Idris MEA, Elfaki OA, Elfakey WEM, Salih KMA. Comparison in the quality of distractors in three and four options type of multiple choice questions. Adv Med Educ Prac. 2017; 8: 287-91.
Ramakrishnan M, Sathe AB, Vinayak A. Item analysis: a tool to increase MCQ validity. Indian J Basic Appl Med Res. 2017; 6: 67-71.
Abdulghani HM, Ahmad F, Irshad M, Khalil MS, Al-Shaikh GK, Syed S, et al. Faculty development programs improve the quality of Multiple Choice Questions items' writing. Sci Rep. 2015; 5: 9556.
Kaur M, Singla S, Mahajan R. Item analysis of in use multiple choice questions in pharmacology. Int JAppl Basic Med Res. 2016; 6: 170-3.
Rush BR, Rankin DC, White BJ. The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Med Edu. 2016; 16: 250.
Namdeo SK, Sahoo B. Item analysis of multiple choice questions from an assessment of medical students in Bhubaneswar, India. International Journal of Research in Medical Sciences. 2016; 4: 1716-9.
Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. J Pak Med Assoc. 2012; 62: 142-7.
Pande SS, Pande SR, Parate VR, Nikam AP, Agrekar SH. Correlation between difficulty and discrimination indices of MCQs in formative exam in physiology. South-East Asian Journal of Medical Education. 2013; 7: 45-50.
Pais J, Silva A, Guimaràƒes B, Povo A, Coelho E, Silva-Pereira F, et al. Do item-writing flaws reduce examinations psychometric quality? BMC Res Notes. 2016; 9: 399.
Park IS, Suh YO, Park HS, Kang SY, Kim KS, Kim GH, et al. Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination. J Educ Eval Health Prof. 2017; 14: 20.
Badkur M, Suryavanshi G, Abraham AK. The correlation between the acceptable range of difficulty and discrimination indices in four response type multiple choice questions in physiology. Indian Journal of Basics and Applied Medical Research. 2017; 6: 695-700.
Tejinder S, Piyush G, Dalijit S. Principles of medical education. 3rded. New Dehli: Jaypee Brothers. 2013. p. 70-1. 15. Hermi A, Achour W. Item analysis of examinations in the Faculty of Medicine of Tunis. Tunis Med. 2016; 94: 247-52.
Borwn FG. Principles of educational and psychological testing. 3rd ed. New York: Rinehart and Winston. 1983. p. 55-6.
Sim SM, RasiahRI. Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singapore. 2006; 35: 67-71.
Kheyami D, Jaradat A, Al-Shibani T, Ali FA. Item Analysis of Multiple Choice Questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos Univ Med J. 2018; 18: 68-74.
Mitra NK, Nagaraja HS, Ponnudurai G, Judson JP. The levels of difficulty and discrimination indices in type a multiplechoice questions of pre-clinical semester 1, multidisciplinary summative tests. IeJSME. 2009; 3: 2-7.
Habib MA, Talukder HK, Rahman MM, Ferdousi S. Post- application Quality Analysis of MCQs of Preclinical Examination Using Item Analysis. Bangladesh Journal of Medical Education. 2016; 7: 2-7.
