Current Pediatric Research

Abstract

Introduction: In Ecuador, newly medical doctors are obliged to do a compulsory rural year to provide services in all medical fields, with a strong emphasis in obstetrics. Maternal mortality ratio has been stabilized around 50 per 100,000 live births, with postpartum hemorrhage as the main cause. Although skills laboratories are now widely used and recommended for undergraduate clinical skills education, medical curricula in Ecuador have little experience using them. As a response to the high perinatal mortality in the country, the Universidad Técnica Particular de Loja (UTPL), in the southeastern part of Ecuador, started a project implementing skills lab training. Objective: The objective of the study was to assess the improvement of the performance in perinatal skills needed in the rural area in final year students after a short educational intervention based on skills lab training. Method: Two subsequent groups of fifth year students were assessed and compared during an Objective Structured Clinical Examination (OSCE). Group 1 consisted of 39 students and Group 2 consisted of 43 students. Both groups received perinatal lessons in regular master classes, the group 2 additionally received training in six perinatal skills in small groups during the semester in the skills lab. Univariate statistics (independent student's t-test or chi square test) was performed to compare basic characteristics in the two groups and to compare final grades after the intervention. A p-value <0.05 was considered statistically significant. Results: Passing students in the total OSCE rises from 11.1% to 42.7% (p<0.05) between the first and the second group. The three stations with “no passing participants” in the first group are: pelvic examination, episiotomy and repair, and postpartum hemorrhage management. The global score for each skill showed an improvement except for Neonatal resuscitation. Conclusion: The implementation of skills lab training facilitates the acquisition and maintenance of perinatal skills. Curriculum changes are needed to improve the student’s competence. A skills laboratory provides a space to develop their skills and to receive an objective assessment.

Keywords

Clinical skills, Perinatal, Ecuador, Medical education, Primary care, Rural health services.

Introduction

In Ecuador, after their graduation, newly medical doctors are obliged to do a compulsory rural year to obtain their license to practice or to go into a postgraduate training [1]. They have to serve in different health care facilities across the country and also provide services in all medical fields, with a strong emphasis in perinatal care, due to the lack of specialists in rural communities, especially in family medicine and obstetrics [2-4]. During this compulsory rural year as “médico rural” (rural doctor), they are exposed to the reality of providing rural health services, including the delivering of more procedures into the obstetrical care area. The transition from medical student to basic doctor in rural place is challenging.

A previous study assessed the self-reported mismatch of obstetrical skills training before graduation in Ecuadorian medical schools and the related tasks required for the compulsory rural year. This research found a gap between the needs of respondents and the training they received [5].

During the last undergraduate year, medicine students receive practical training in the teaching hospitals in five medical services: internal medicine, surgery, obstetrics and gynecology, pediatrics and pre-rural. During this year they are called “internos rotativos”, and this is the last stage for them to learn and practice under supervision. It is assumed that if the students spend time in these rotations they will acquire the knowledge and the skills they will need it [6-9].

In recent years some universities from United States of America, have been implementing checklists to facilitate the documentation of clinical experiences of students during their rotations as well as to research in skills development during medical training, in rural areas rotations or in junior doctors [10,11]. Even though such improvements are in place, there is little information about the best way to train students to work as rural doctor with focus on perinatal skills during the undergraduate clinical attachments in Ecuador [12,13].

Skills laboratories are now widely used and recommended for undergraduate clinical skills education. Their value for improving the systematic approach, the completeness and proficiency of basic clinical skills has often been demonstrated [14-17]. Medical education in Ecuador has little experience using skills laboratories for undergraduate medical training as other developing countries as Vietnam or Pakistan [18,19]. Therefore in 2012, the Faculty of Medicine of the Universidad Técnica Particular de Loja (UTPL), in the southeastern part of Ecuador, started a project to investigate the added value of a skills laboratory in their curriculum. In this project a specific strategy was designed for obstetrical skills as a response to the high perinatal mortality in this country that has been stabilized around 50 per 100,000 live births [20].

The aim of the study was to know the effect of a short educational intervention, including skills lab training on the performance in perinatal skills of undergraduate medical students.

Methodology

Subjects

In this study design two subsequent promotions of fifth year students were assessed during two days; one of them had training in a recently set up skills laboratory. An Objective Structured Clinical Examination (OSCE), a validated tool to objectively assess a range of clinical skills under similar circumstances through various stations was used [21,22].

Group 1 consisted of 39 students (of 41 from a fifth year students´ class of 2013) who received obstetrics in regular master classes without any formal practice and who graduated in 2013. The students were assessed during their final month before the end of the year period. Prior to the assessment, they received verbal instructions on the OSCE organization because they were not familiar with it.

Group 2 consisted of 43 students, one year later (of 46 from a fifth year students´ graduated in 2014). They also received obstetrics in regular master classes and additionally they received training in the skills lab, in six obstetrical skills in small groups during the semester. The first training group was three months before the evaluation and the last group was three weeks before the OSCE. The brief intervention consisted of six sessions, each one of two hours of duration with eight tutors.

In both groups the scores of the OSCE were not taken into account in their regular assessment to avoid tension and fear in the students.

Proceedings of the OSCE Development

The design of the OSCE comprised stations to evaluate medical students´ perinatal clinical skills. The development of the stations was preceded by debates with UTPL faculty of Obstetrics and Gynecology. The blueprint of the OSCE was framed with respect to the learning objectives of the basic doctor to work in rural places and the maternal mortality causes in the country. All scripts passed a revision process. In total, six scenarios were written and these covered the content of the training sessions in the skills lab. The UTPL faculty of Obstetrics and Gynecology approved the scenarios, the organization of the OSCE and the checklists for assessment.

The six OSCE stations were: taking blood pressure; pelvic examination with cytology test smear; labor delivery control and partogram use; episiotomy and repair; postpartum hemorrhage management; and, new born resuscitation.

The necessary materials were provided for every action in the stations, including an artificial manikin; only the blood pressure station used simulated patients. Each station had duration of 12 min. 1 min was given to facilitate switching between stations. The scenarios of the stations were exactly the same for every student from each group. All students completed the circuit over an 80 min period.

The OSCE checklists had two parts: the first one adopted a simple “yes” or “no” recording scheme with each checklist item, representing one or no points respectively. The checklists contained 19 to 28 items per station including communication, knowledge and procedural skills. The second part used three global scores for completeness, systematic approach and proficiency, based on literature. Completeness was defined as completed all the statements; systematic approach as methodical in procedure; and, proficiency as advancement in skill [17,23]. These scores were given on a five-point scale representing bad, regular, good, very good and excellent respectively.

To pass a station, a cut off score of at least 70% of correct checklist items was used conform the university rules on assessment. For the stations “taking blood pressure” and “postpartum hemorrhage” it was necessary to have at least eighteen correct items, for “pelvic examination” nineteen; for “labor delivery control” twenty; for “episiotomy and repair” fifteen; and for “new born resuscitation” fourteen. A score was calculated for systematic approach, completeness, and proficiency for each station.

The observers were different from the ones who trained the students in the skills labs. They received training to observe and to fill out the checklists in each one of the stations one day before the OSCE. The observers assigned checks to the student for each correct item in the score list and used the three global scores at the end of each assessment. They were also trained to award a global rating for systematic approach, completeness and proficiency.

All the instructions were described in the “instructions for observers” booklet. During the first OSCE (group 1) four of the examiners were family physicians and three were basic doctors with experience in teaching at the UTPL. In the second OSCE (group 2) the observers were two surgeons, and four basic doctors with experience in teaching at UTPL.

Statistical Analysis

SPSS (V.20.0.0) was used for statistical analysis. Univariate statistics (independent student’s t-test or chi square test) was performed to compare basic characteristics in the two groups and to compare final grades after the intervention. A z-test was used to compare two sample proportions. A p-value <0.05 was considered statistically significant.

Results

Both groups were comparable for their basic characteristics, such as age, gender and grades (Table 1). They were all Ecuadorian students in the UTPL.

Scores of the items that were tested are provided in Table 2. On average, the amount of passing students in the total OSCE rises from 11.1% to 42.7% between the first and the second group. The three stations with “no passing participants” in the first group are: pelvic examination, episiotomy and repair and postpartum hemorrhage management.

Table 3 shows the scores on completeness, systematic approach and proficiency per skill. The global score for each skill showed an improvement except for Neonatal resuscitation.

Tables 4 and 5 show the subgroup analysis of students by grade of complete the fifth year, the weakest group (70-85% of points) and the strongest group (86-100%). In a post-hoc subgroup analysis, the amount of passing students rises from 8.3% to 38.5% weakest students; in the strongest students there is a similar increase from 13.3% to 47.1%.

Table 1. Basic characteristics of the students in the two groups

Table 2. OSCE results of the six obstetrical skills in two groups of fifth year students. A pass is defined as at least 70 percent of items correct

Table 3. Means of systematic approach, completeness and proficiency in the OSCES, maximal score 5.

Table 4. OSCE results of the six obstetrical skills in two groups of students who passed their 5th year with 70-85% of points (“weakest group”)

Table 5. OSCE results of the six obstetrical skills in two groups of students who passed their 5th year with 85-100% of points (“strongest group”)

Discussion

This is the first study in Ecuador to test a brief educational intervention in undergraduate students to improve the performance of perinatal skills which are important to practice in rural Ecuador. It is also the first time that educational benefits of a skills laboratory were evaluated in this country. The results show that the first cohort without skills laboratory training scored significantly less (11%) than the second group with skill laboratory training (43%). A significant increase was observed for the item scores and for the global scores. The main conclusion therefore is that the performance of these skills can be significantly better when using training sessions in the skills laboratory, compared to giving only master classes. This positive effect of skills lab training was identified in other studies in undergraduate and postgraduate students [17,18,24-27].

It is obvious that the students have to be familiar with these perinatal skills previous to the internship and rural compulsory year. In the traditional curriculum without the skills lab, the students are not exposed to these skills during the clinical attachments [28,29].

The implementation of skills lab training facilitates the acquisition and maintenance of obstetrical skills that are needed in the rural year [18,30]. The students can practice skills repetitively with no inconvenience to patients or risk to their safety [14,26,29]. It allows last year students to face the patient with certainty and confidence in the real context of rural medicine practice [31-34].

In Ecuador there are only a few experiences using manikins in interactive workshop and skills training in obstetrics. One of these is the Advance Life Support in Obstetrics–ALSO® Course implanted in Ecuador since 2003 [35]. In the faculty, the results of this study support the need to have a broader spectrum of training and assessment methods in the curriculum, especially in reference to judgment of professional services done by a successful physician [36]. Curriculum changes have to be made to improve the student’s competence. This emphasizes in make explicit the clinical skills curriculum to provide the opportunity for the student to master it [9]. Training needs should be related to country morbidity and mortality causes, and the required standard practice for graduates focused on outcomes [18,37-39]. Faculty staff and officials may be urged to chance old curricular habits using objective findings about their students [15,16,40]. The review of group performances in an OSCE is helpful in demonstrating to the faculty the areas of weakness in the educational program [32].

There is no significant difference between the results of the OSCE evaluation and the promotion grade of the students. The effect of the intervention seems to be greater in the weakest group, but the power is too low to compare both groups statistically. It is known that the best predictor of effectiveness is the mental ability and that clinical performance is not generally predicted by academic scores. We did not research on the empathy and motivation to be a doctor as factors that predict clinical and academic success [41]. Learned information is quickly lost down without practice. The courses given do not follow the same curricula in the traditional classes [42]. It could be another factor. Future research is needed to evaluate the impact of this course on successful and unsuccessful students.

Further research is necessary to evaluate students’ perceptions of this kind of training and evaluation. During the final meeting some students gave feedback and they are satisfied with the OSCE. As mentioned by others, the students felt that they need more skills training to function better in the rural areas [30]. It is well known that the OSCE examination results in improved feedback to students and staff [31]. Also is necessary to evaluate the retention of the skills improvement in a period of time and another aspects as the correct documentation after the event, team working, communication, interdisciplinary relationships [26,38].

The improvement of these skills may help to reduce Ecuador´s maternal mortality ratio that has been stabilized around 50 per 100,000 live births and it´s first cause of maternal mortality: postpartum hemorrhage, which is in the 8th position compared with Latin American countries (from less to more) and more than 10 times as frequent as compared to European countries like Belgium [20,43].

Limitations of the Study

A limited number of skills were tested using an OSCE. This is still not an “in the field” situation and the real effect of the use of obstetrical skills during the rural years of the graduates needs to be assessed in the rural practices themselves.

It is unlikely that information was shared between the subsequent group’s students or even if it was, that it had no significant effect on scores during the two days evaluation [32]. It was the first time that the observers were scoring an OSCE, although they were trained before, this could lead to a bias in the students’ evaluation. Also the observers were different in the two groups. However, other studies show that senior clinicians and faculty are reliable for the evaluation of clinical competences [17]. The use of fixed scoring lists and the standardisation of the OSCE stations have limited the possible bias.

Implications for other Medical Schools in Mid Income Countries

The results of this study are from a private University located in the southeastern part of Ecuador with a limited number of medical students. In this university, the skills lab was established successfully based on the findings of this report. More studies and also incentives are needed to implement skills lab training in other universities.

Tran mentioned that to improve the skills lab it is necessary to integrate it into the existing curriculum, to increase time allocated for training certain skills, to ensure models/manikins are appropriate for the training, and to reconsider the requirements of students for each skill in the OSCE [18].

A possible problem when other faculties want to implement a skills lab are, among other, personal commitment of one or two faculty members that engage in the project, available faculty time and allocation of premises [32]. However, worldwide innovative teaching methods are being implemented and medical school in these countries should be aware of this [44,45].

In South Africa the Health Department is trying to train in-service the existing health workers to equip them with skills to render the required service needs. It remains the question for the recent graduates, what their outcomes should be [46].

It is necessary to take into account that the undergraduate students still need regular training after their graduation to maintain the obstetrical skills competences, more if they will work in rural places [26].

Conclusion

Introducing perinatal skills training resulted in an important increase of skills in the students, compared to the basis master classes only [25]. The training in the skill laboratory and OSCE evaluation is a favorable area for the development of medical education in Ecuador and other mid-income countries. A skills laboratory proposes a space for students to develop their skills and to receive an objective assessment [18].

The culture in our faculty towards more hands on training increases. Perinatal skills are only one part of skills needed in the rural practice. The challenge is to implement clinical skills of other domains in a skills laboratory setting.

Acknowledgement

The authors thank Jana Bobokova, former dean of the Faculty of Medicine at UTPL for her support and the entire faculty involved in the process of the development of the objective structured clinical examination and especially Diego Gomez who arranges the administrative issues at the university.

Source of Funding

The work is supported by VLIR-UOS Project ZEIN2010PR377 ‘Improving and strengthening quality for Family Medicine training in Ecuador, using capacity building and distant learning’ developed between University of Antwerp, Belgium and Universidad Técnica Particular de Loja, Ecuador.

References

1. Decreto Supremo No 44. Registro Oficial Nro 13. Ecuador; 1970.
2. Glazebrook R, Harrison S. Obstacles to maintenance of advanced procedural skills for rural and remote medical practitioners in Australia. Rural Remote Health 2006; 6: 502.
3. Gaus D, Herrera D, Heisler M, et al. Making secondary care a primary concern: The rural hospital in Ecuador. Revista Panamericana de Salud Pública 2008; 23: 212?217.
4. Strasser R, Neusy AJ. Context counts: training health workers in and for rural and remote areas. Bull World Health Organ 2010; 88: 777?782.
5. Sánchez Del Hierro G, Remmen R, Verhoeven V, et al. Are recent graduates enough prepared to perform obstetric skills in their rural and compulsory year? A study from Ecuador. BMJ Open. 2014; 4: e005759.
6. Pontificia Universidad Católica del Ecuador. Malla Curricular de la carrera de Medicina. Quito, Ecuador 2008.
7. Universidad Técnica Particular de Loja. Pensum Carrera de Medicina. Loja, Ecuador 2010.
8. Carrillo J, Delgado B, Kosik RO, et al. Medical education in Ecuador. Med Teach 2013; 35: 979?984.
9. Danoff D, Corbett EC, Fortin AH, et al. Recommendations for clinical skills curricula for undergraduate medical education. Washington, D.C: Task Force on the Clinical Skills Education of Medical Students. Association of American Medical Colleges 2005: 40.
10. Hunsaker M, Glasser M, Neilsen K, et al. Medical students?assessments of skill development in rural primary care clinics. Rural Remote Health 2006; 6: 616.
11. Connolly A, Davis K, Casey P, et al. Multicenter trial of the clinical activities tool to document the comparability of clinical experiences in obstetrics-gynecology clerkships. Acad Med 2010; 85: 716?720.
12. College of Family Physicians of Canada. Society of Rural Physicians of Canada. Society of Obstetricians and Gynaecologists of Canada. Joint position paper on training for rural family practitioners in advanced maternity skills and cesarean section. Canadian family physician Médecin de famille canadien 1999; 45: 2416-2432.
13. Gaus DP. The rural hospital in ecuador. Health Affairs 2009; 28: 1003?1010.
14. Corbett EC, Berkow R, Bernstein LB, et al. Recommendations for preclerkship clinical skills education for undergraduate medical education. Washington, D.C.: Task force on the clinical skills education of medical students. Association of American Medical Colleges 2008: 39.
15. Remmen R, Scherpbier A, van der Vleuten C, et al. Effectiveness of basic clinical skills training programmes: A cross-sectional comparison of four medical schools. Medical education 2001; 35: 121?128.
16. Peeraer G, Scherpbier a JJ a, Remmen R, et al. Clinical skills training in a skills lab compared with skills training in internships: comparison of skills development curricula. Educ Health (Abingdon) 2007; 20: 125.
17. Hendrickx K, De Winter B, Tjalma W, et al. Learning intimate examinations with simulated patients: The evaluation of medical students? performance. Med Teach 2009; 31: e139-47.
18. Tran T, Scherpbier A, van Dalen J, et al. Implementing a skillslab training program in a developing country. Educ Health (Abingdon) 2014; 27: 243.
19. Iqbal M, Khizar B, Zaidi Z. Revising an objective structured clinical examination in a resource-limited Pakistani Medical School. Educ Health (Abingdon) 2009; 22.
20. Instituto Nacional de Estadística y Censos. Anuario de estadísticas vitales nacimientos y defunciones 2014. Quito 2014.
21. Patricio M, Juliao M, Fareleira F, et al. A comprehensive checklist for reporting the use of OSCEs. Med Teach 2009; 31: 112?124.
22. Gormley G. Summative OSCEs in undergraduate medical education. Ulster Med J 2011; 80: 127?132.
23. Peeraer G, Muijtjens AMM, De Winter BY, et al. Unintentional failure to assess for experience in senior undergraduate OSCE scoring. Medical Education 2008; 42: 669?675.
24. Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: A comparison of self-reported confidence and observed competence. Medical Education 2004; 38: 358?367.
25. Lynagh M, Burton R, Sanson-Fisher R. A systematic review of medical skills laboratory training: Where to from here? Medical Education 2007; 41: 879?887.
26. Cass GKS, Crofts JF, Draycott TJ. The use of simulation to teach clinical skills in obstetrics. Semin Perinatol 2011;35: 68-73.
27. Fisher N, Bernstein PS, Satin A, et al. Resident training for eclampsia and magnesium toxicity management: simulation or traditional lecture? Am J Obstet Gynecol 2010; 203: e1-5.
28. Remmen R, Derese A, Scherpbier A, et al. Can medical schools rely on clerkships to train students in basic clinical skills? Medical Education 1999; 33: 600?605.
29. Baeza AMA, Mella BJ, Soldati OA, et al. Aplicabilidad de simuladores de mediana complejidad en el proceso de formación de residentes de ginecologia y obstetricia. Revista chilena de obstetricia y ginecología 2010; 75: 349?354.
30. Huang YS, Liu M, Huang CH, Liu KM. Implementation of an OSCE at Kaohsiung Medical University. Kaohsiung J Med Sci 2007; 23: 161-169.
31. Harden R, Stevenson M, Downie W, Wilson G. Assessment of clinical competence using objective structured examination. Br Med J 1975; 1: 447?451.
32. Carraccio C, Englander R. The objective structured clinical examination: A step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med 2000; 154: 736?741.
33. Halaas GW, Zink T, Brooks KD, Miller J. Clinical skills day: Preparing third year medical students for their rural rotation. Rural Remote Health 2007; 7: 788.
34. Casey PM, Goepfert AR, Espey EL, et al. To the point: reviews in medical education--the objective structured clinical examination. Am J Obstet Gynecol 2009; 200: 25?34.
35. Dresang LT, Rodney WM, Leeman L, et al. Advanced life support in obstetrics in Ecuador: Teaching the teachers. J Am Board Fam Pract 2004; 17: 276?282.
36. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65: S63-67.
37. Al-Elq MA. Medicine and clinical skills laboratories. J Fam Community Med 2007; 14: 1-8.
38. Homaifar N, Mwesigye D, Tchwenko S, et al. Emergency obstetrics knowledge and practical skills retention among medical students in Rwanda following a short training course. Int J Gynaecol Obstet 2013; 120: 195-199.
39. Englander R, Flynn T, Call S, et al. Toward defining the foundation of the MD degree: Core entrustable professional activities for entering residency. Acad Med 2016; XX: 1-7.
40. Peeraer G, De Winter BY, Muijtjens MM, et al. Evaluating the effectiveness of curriculum change. Is there a difference between graduating student outcomes from two different curricula? Med Teach 2009; 31: e64-68.
41. Hughes P. Can we improve on how we select medical students? JRSM 2002; 95: 18-22.
42. Augustin M. How to learn effectively in medical school: test yourself, learn actively and repeat in intervals. Yale J Biol Med 2014; 87: 207-212.
43. WHO, UNICEF, UNFPA, The World Bank, The United Nations Population Division. Trends in maternal mortality: 1990 to 2013.
44. Karle H. Global standards and accreditation in medical education: A view from the WFME. Acad Med 2006; 81: S43-48.
45. Zhang Q, Lee L, Gruppen LD, Ba D. Medical education: Changes and perspectives. Med Teach 2013; 35: 621?627.
46. Schoon MG, Motlolometsi MWA. Poor maternal outcomes: A factor of poor professional systems design. S Afr Med J 2012; 102.