Assessment of collaborative problem solving skills

Original Article Assessment of collaborative problem solving skills in Undergraduate Medical Students at Ziauddin College of Medicine, Karachi Arsalan Manzoor Mughal1, Sirajul Haque Shaikh2 ABSTRACT Objective: Collaborative Problem Solving Empirical Progressions from the Assessment and Teaching of 21st Century Skills (ATC21S) framework were used to determine the level of collaborative problem solving skills (CPS) in first, second and third year MBBS students at Ziauddin College of Medicine during Problem-Based Learning (PBL) sessions. Variations based on gender and roles were studied. Methods: It is an analytical comparative cross-sectional study in which seven PBL groups were selected per year by non-probability convenient sampling. Data was collected using the Collaborative Problem Solving Five Strands Empirical Progressions by the primary investigator through observation of the students during PBL sessions. Duration of study was six months. Results: We found that in our students, development of social dimension skills is facilitated to a greater extent than the development of cognitive dimension skills through the process of PBL. These skills are generally better developed in the leader compared to the scribe and members in a group. They are also more developed in females compared to males. Modification in them is also observed as the year’s progress. Conclusion: Although PBLs facilitate development of CPS skills’ progression however in our curriculum, PBLs mainly focus on social skills development and have less emphasis on cognitive skill development. Thus, hybrid instructional strategies with components from TBL and mentorship are recommended for better development of CPS skills. KEYWORDS: Problem-Based Learning, Problem Solving. doi: https://doi.org/10.12669/pjms.341.13485 How to cite this: Mughal AM, Shaikh SH. Assessment of collaborative problem solving skills in Undergraduate Medical Students at Ziauddin College of Medicine, Karachi. Pak J Med Sci. 2018;34(1):185-189. doi: https://doi.org/10.12669/pjms.341.13485 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. Arsalan Manzoor Mughal, Assistant Professor, INTRODUCTION Department of Anatomy, Ziauddin University, Although traditional disciplines such as basic Karachi, Pakistan. 2. Sirajul Haque Shaikh, and clinical sciences have their place in medical Director, education, there has been a shift towards the Department of Medical Education, teaching of additional soft skills which will help the College of Physicians & Surgeons, Karachi, Pakistan. doctor of tomorrow work in a broader and more Correspondence: connected workplace.1 These are called 21st century skills.2 Despite the recognition of their significance, Arsalan Manzoor Mughal, Assistant Professor, only a few structured frameworks are available for Department of Anatomy, assessment and development of these skills.3 The Ziauddin University, Karachi, Pakistan. Assessment and Teaching of 21st Century Skills Email: [email protected] project (ATC21S) with contributions from 250 researchers from around the world is one of the * Received for Publication: July 11, 2017 * Revision Received: November 17, 2017 most comprehensive among these frameworks. It * Second Revision Received: December 19, 2017 provides comprehensive tasks for the assessment of * Final Revision Accepted: December 23, 2017 21st century skills.4 One of these skills is Collaborative Pak J Med Sci January - February 2018 Vol. 34 No. 1 www.pjms.com.pk 185  Arsalan Manzoor Mughal et al. Problem Solving (CPS) which is defined as “working batch is divided into 13 PBL Groups with 10 students with others to solve a common challenge, including in each to conduct PBLs. A PBL comprises of two the contribution and exchange of ideas, knowledge Sessions each of two hours duration. Learning or resources in order to achieve a shared goal”.5 CPS goals are generated at the end of Session one by are ranked among the top five most essential skills completing Jumps 1-5 in a small group discussion required for success in college and the workforce.6 format. These are researched by the students during According to the ATC21S project, CPS can be self-study sessions as part of Jump 6. In Session two, classified into two dimensions - social skills and students collaborate to discuss and achieve the set cognitive skills. The three elements of social skills goals as part of Jump 7. Each session is facilitated by include participation, perspective taking and social an instructor who is trained to conduct the process. regulation whereas the two elements of cognitive Students are given roles during each session which skills are task regulation and knowledge building. include a leader, who directs the group, and a These are called the five strands of CPS.7 scribe, who takes notes. The remaining participants Participation skills include the level of action, are called the members. These roles are rotational interaction and task completion. Perspective and change with each PBL. taking skills include responsiveness and audience Despite the claim that PBLs help develops CPS awareness. Social regulation skills include skills, no comprehensive framework was previously metamemory, transactive memory, negotiation available to assess these skills. So, we used the and initiative taking. Task regulation skills ATC21S framework Collaborative Problem Solving include problem analysis, goal setting, resource Five Strands Empirical Progressions to determines management and planning complexity. Knowledge the level of these skills in first to third year MBBS building skills includes systematicity and students and to find variations based on gender and information acquisition.8 roles. The progression of each elemental CPS skill can be graded from A to F using the five-strand empirical METHODS projection frame work7 by observing the student in a collaborative problem-solving process. This In this comparative cross sectional study, seven assessment indicates the extent of development of PBL groups were selected per year by non-probability each CPS skill element.9 convenient sampling for adequate representation Problem Based Learning (PBL)is an instructional at a confidence interval of 95%, confidence level method developed by Howard Barrows and Victor of 5%. After written informed consent, data was Neufeld at McMaster University, in which patient collected for each student individually regarding problems are used by the students to learn about the social skills (participation, perspective taking, basic and clinical sciences.10 Advantages of the PBL social regulation) in Session One and cognitive process include development of inter-disciplinary skills (task regulation, knowledge building) in thinking, solution of complex tasks and reduction of Session Two using the five-strand empirical cognitive overload by scaffolding provided by the projection framework7 by observing the student in facilitator.11 In addition, PBLs also facilitate learning each collaborative problem-solving session. Each of additional essential skills such as self-directed CPS skill was graded from A to F in the order of learning, epistemic practices and collaboration.11 development as described in the framework. All Though these skills are not examined in tests, they data was collected by the primary investigator. are thought to be important for lifelong learning.11 Ethical approval was obtained from Ethics However due to the lack of a proper assessment review committee of Ziauddin University prior to framework, it has also been claimed that it is commencement of study. unlikely that problem solving skills exist or if they Categorical data of CPS grades A-F was converted can ever be learned by PBLs.12 to numerical scores 1-6 respectively. Data was The Ziauddin College of Medicine has an entered into SPSS and analysed by SPSS statistical integrated modular hybrid curriculum with a mix software. 2-tailed independent samples student t of lectures, lab sessions as well as problem based test was used to compare mean scores of genders sessions in the undergraduate medical program. whereas analysis of variance (ANOVA) was used It was the first college in the country to introduce to compare mean scores of years 1-3 and roles of PBLs back in 1996.13 The seven Jumps PBL process students. P-value less than 0.05 was considered as is carried out through MBBS years 1-3. Each MBBS significant. Pak J Med Sci January - February 2018 Vol. 34 No. 1 www.pjms.com.pk 186  Assessment of collaborative problem solving skills Table-I: Mean and Standard Deviation of Collaborative Problem Solving Skills Scores. Collaborative Problem Solving Skills N Mean Standard Deviation Social Skills Participation 210 3.07 1.316 Perspective Taking 210 3.09 1.347 Social Regulation 210 3.10 1.364 Cognitive skills Task Regulation 210 3.00 1.334 Knowledge Building 210 2.92 1.326 RESULTS in all five CPS skills were significantly higher in the first year MBBS batch and were progressively Overall, the mean scores of the students were lower in the second and third year batches. The higher in social skill compared to cognitive skills. strongest skills in first year and third year students They scored highest in social regulation followed were perspective taking, social regulation and by perspective taking, participation, task regulation participation respectively. Knowledge building and knowledge building (Table-I). was the weakest skill in all three batches (Fig.3). There was a significant difference between the performance of leaders, scribes and members in all DISCUSSION four skills except knowledge building. Mean scores Collaborative problem solving is an essential skill were generally higher for the leaders compared to for the doctor of tomorrow.14 It is evident by the the scribes and members (Fig.1). fact that collaboration and communication skills A comparison of performance based on gender have been added to most competency frameworks indicates that females got significantly higher scores of medical schools.15 Problem based learning is in all skills compared to males. Among males, the believed to develop these skills however to the best strongest skill was perspective taking whereas of our knowledge; this is the first study to assess females scored highest in social regulation (Fig.2). these skills in our local context. A comparison between performances of different We found a mean score of around three in year batches indicates that the mean scores obtained social and cognitive skills indicating “awareness of partnership” and “sharing & connecting information” in our student’s respectively. The Fig.1: Comparison of mean collaborative problem Fig.2: Comparison of mean collaborative problem solving skills scores in leader, scribe and member. solving skills scores in genders. Error bars indicate 95% confidence interval. Error bars indicate 95% confidence interval. * indicates p-value<0.05. * indicates p-value<0.05. Pak J Med Sci January - February 2018 Vol. 34 No. 1 www.pjms.com.pk 187  Arsalan Manzoor Mughal et al. social skills can be encouraged in males by reflection in mentorship programs.21 The significantly better performance of first year batch in all CPS skills domains and progressive decline in the subsequent years is worth noting. The reason for this could be a known problem that in a hybrid curriculum, there is inherent conflict with lectures which causes knowledge interference.22 In our college, lectures are taken by the senior faculty with greater subject knowledge and the PBLs are facilitated by the junior faculty with relatively less knowledge depth. Al Haqwi et al.23 suggests that the facilitator should possess both content and process expertise to get the best results from the PBL process. As the older students are self-regulated learners, they find lectures, demonstration and lab sessions more useful in this regard. Significance of traditional instructional strategies in this regard has also been reported by other studies from the Fig.3: Comparison of mean collaborative problem solving skills scores in MBBS Year 1-3. region.24 Error bars indicate 95% confidence interval. The progressive change in the mean social skills * indicates p-value<0.05. score of batches indicates their modification with time. We noted a higher mean score of perspective higher mean social skills scores in his study indicate taking in the first-year batch, indicating their effort that the process of PBL is encouraging their use. to understand each other’s points of view. The Ahonen et al.16 also reported that students ranked second-year batch has a higher mean score in social social skills and collaboration highest among the regulation, indicating their ability to resolve conflict. 21st century skills. However, comparatively weaker The third-year batch has a higher mean score in scores in cognitive skills in our study highlights the participation, indicating uniform contribution from relative weakness of the process in this dimension. the group. Social skills development is essential as Multiple studies have also reported that PBLs process conflicts often turn into relationship conflicts have no greater benefit in short term knowledge when social skills are weak.25 Thus improved social based test performance compared to traditional teaching methods.17,18 A recommendation could be skills enhance collaboration. use of additional components such as mandatory Limitations of the Study: As this study was preassigned readings with individual and conducted in a single medical college, the results collaborative test solving. These are commonly may reflect a local perspective and may be difficult used in Team Based Learning (TBL).19 to generalize. Also, being an observational study, Higher mean scores of CPS skills, especially in the social domain by the leaders compared to the the element of subjective bias cannot be ruled out. scribes and members indicates greater use of CPS However, it was minimized by the fact that the skills during the PBL. This is extremely positive as data collection tool was highly structured and all each student is given a chance on rotation to be a data was collected by a single primary investigator group leader, thereby encouraging them to use who has an 8 years’ experience of generating and these skills. These skills will equip the students conducting PBL sessions at the university and is a with the ability to manage and improve health care student of medical education. We couldn’t generate systems with a clear vision in a leadership role.20 a comparative group which is not exposed to PBL We found better CPS skills scores in females because all medical students undergo PBL teaching compared to males, both in social and cognitive at the medical college in the basic health sciences. domains. Such gender differences have also been Only year 1-3 students could be included as the reported by other studies. Ahonen et al.16 reports fourth and final year students don’t have PBLs. that girls ranked social skills more highly whereas Knowing the levels of CPS skills in our students boys preferred technical skills. Development of and their progression will help plan better PBLs Pak J Med Sci January - February 2018 Vol. 34 No. 1 www.pjms.com.pk 188  Assessment of collaborative problem solving skills and other teaching strategies to enhance their 12. Schmidt HG, Rotgans JI, Yew EH. The process of problem-based learning: what works and why. Med Educ. 2011;45(8):792-806. development. Teacher awareness and training will 13. Huda N. PBL a fad or reality? J Pak Med Assoc. 2004;54(4):168- be essential in this regard.26 169. 14. Combes JR, Arespacochaga E. Physician competencies for a 21st CONCLUSION century health care system. J Grad Med Educ. 2012;4(3):401-405. 15. Toward a Common Taxonomy of Competency Domains for PBLs facilitate development of CPS skills the... : Academic Medicine [Internet]. LWW. [Cited 2017 Jun 29]. Available from: http://journals.lww.com/academicmedicine/ progression, as our students on average were Fulltext/2013/08000/Toward_a_Common_Taxonomy_of_ aware of their partners and were able to share and Competency_Domains_for.21.aspx. connect information. It also facilitates development 16. Ahonen AK, Kinnunen P. How do students value the importance of twenty-first century skills? Scand J Educ Res. 2015;59(4):395-412. in leadership skills in the role of group leaders. 17. Pourshanazari AA, Roohbakhsh A, Khazaei M, Tajadini H. However, in a hybrid curriculum, PBLs mainly Comparing the long-term retention of a physiology course focus on social skills development whereas for medical students with the traditional and problem-based learning. Adv Health Sci Educ. 2013;18(1):91-97. traditional teaching methodologies retain their 18. Kirschner PA, Sweller J, Clark RE. Why minimal guidance significance in cognitive skill development. Also, during instruction does not work: An analysis of the failure females have higher CPS skills compared to males in of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educ Psychol. 2006;41(2):75-86. these PBL sessions. Hybrid instructional strategies doi: 10.1207/s15326985ep4102_1. with components from TBL and mentorship are 19. Dolmans D, Michaelsen L, van Merriënboer J, van der Vleuten recommended for better development of CPS skills. C. Should we choose between problem-based learning and team-based learning? No, combine the best of both worlds! Med Declaration of Interest: We declare no conflict of Teach. 2015;37(4):354-359. doi: 10.3109/0142159X.2014.948828. 20. A First Step Toward Understanding Best Practices in Leadersh... : interest. Academic Medicine [Internet]. LWW. [Cited 2017 Jun 15]. Available from: http://journals.lww.com/academicmedicine/ REFERENCES Fulltext/2014/11000/A_First_Step_Toward_Understanding_ Best_Practices.39.aspx. 1. Levenson R, Atkinson S, Shepherd S. Understanding the doctors 21. Mat Nor MZ, Yusoff MSB, Abdul Rahim AF. Characteristics of of tomorrow. Available from: https://www.kingsfund.org. mentoring programmes in the early phase of medical training at uk/sites/files/kf/field/field_publication_file/Understanding- the Universiti Sains, Malaysia. J Taibah Univ Med Sci [Internet]. doctors-tomorrow-levenson-atkinson-shepherd-oct10.pdf. Available from: http://www.sciencedirect.com/science/article/ 2. Metz S. 21st-Century Skills. Sci Teach. 2011;78(7):6. pii/S1658361217300252. 3. Dede C. Comparing frameworks for 21st century skills. 21st 22. Mansur DI, Kayastha SR, Makaju R, Dongol M. Problem Based Century Ski Rethink Stud Learn. 2010;20:51-76. Learning in Medical Education. Kathmandu Univ Med J. 4. Voogt J, Roblin NP. A comparative analysis of international 2014;10(4):78-82. frameworks for 21st century competences: Implications for 23. AlHaqwi AI. Learning Outcomes and Tutoring in Problem national curriculum policies. J Curric Stud. 2012;44(3):299-321. Based-Learning: How do Undergraduate Medical Students doi: 10.1080/00220272.2012.668938. Perceive Them? Int J Health Sci. 2014;8(2):125-132. 5. Binkley M, Erstad O, Herman J, Raizen S, Ripley M, Miller- 24. Tayyeb R. Effectiveness of problem based learning as an Ricci M, et al. Defining Twenty-First Century Skills. In: Griffin instructional tool for acquisition of content knowledge and P, McGaw B, Care E, editors. Assessment and Teaching of 21st promotion of critical thinking among medical students. J Coll Century Skills. Springer Netherlands. 2012:17-66. Physicians Surg Pak. 2013;23(1):42-46. 6. Oliveri ME, Lawless R, Molloy H. A Literature Review on 25. Collaboration, intragroup conflict, and social skills in Collaborative Problem Solving for College and Workforce project-based learning [Internet]. ResearchGate. [cited 2017 Readiness. ETS Res Rep Ser [Internet]. 2017; Available from: Jun 15]. Available from: https://www.researchgate.net/ http://onlinelibrary.wiley.com/doi/10.1002/ets2.12133/full. publication/276883761_Collaboration_intragroup_conflict_ 7. Emprirical Progressions [Internet]. Assessment & Teaching of and_social_skills_in_project-based_learning. 21st Century Skills. [Cited 2015 Jun 10]. Available from: http:// 26. Häkkinen P, Järvelä S, Mäkitalo-Siegl K, Ahonen A, Näykki P, www.atc21s.org/emprirical-progressions.html. Valtonen T. Preparing teacher-students for twenty-first-century 8. Assessment and Teaching of 21st Century Skills - Methods and | learning practices (PREP 21): a framework for enhancing Patrick Griffin | Springer [Internet]. [Cited 2017 Jun 29]. Available collaborative problem-solving and strategic learning skills. from: http://www.springer.com/gp/book/9789401793940. Teach Teach. 2017;23(1):25-41. 9. Professional Development Modules [Internet]. Assessment & Teaching of 21st Century Skills. Available from: http://www. Authors’ Contribution: atc21s.org/professional-development-modules.html. 10. Swanwick T. Understanding Medical Education: Evidence,Theory and Practice. 2 edition. Chichester, West AMM: Conception of research, acquisition, analysis Sussex: Wiley-Blackwell; 2013. 520 p. and interpretation of data. 11. Hmelo-Silver CE, Duncan RG, Chinn CA. Scaffolding and SHS: Critical revision for important intellectual achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educ Psychol. content, final approval of the version to be 2007;42(2):99–107. published. Pak J Med Sci January - February 2018 Vol. 34 No. 1 www.pjms.com.pk 189