Rabea Krings¹, Sabine Feller¹, Kai Schnabel¹, Sabine Kroiss³, Maja Steinlin², Sören Huwendiek^1
1University of Bern, Switzerland; ²University Children’s Hospital, Inselspital, University of Bern; ³University Children’s Hospital Zurich – Eleonorenstiftung; rabea.krings@iml.unibe.ch
Background: Assessment drives Learning (Cilliers et al. 2010). To enhance the educational effect of learning the handling of children as patients, it is necessary to have appropriate education, but also a realistic assessment method. There is little research concerning children as standardized patients inside this assessment process. Therefore, the goal of the present study is to evaluate perception, acceptance, fairness, and feasibility of an OSCE including children. Furthermore, the educational impact for students was analyzed.
Methods: The regular summative OSCE for fifth year medical students in Bern took place on six consecutive half days in April 2016. 191 students were tested in nine different OSCE stations – elementary school children were engaged as standardized patients for the pediatric station. With regard to Darling and Bardgett (2013), by individual interviews, children were asked afterwards which aspects they liked and disliked in taking part in the Paediatric OSCE. Moreover, the usefulness of children in an OSCE was analyzed with concern to realism, fairness, feasibility, and acceptance. In addition, the educational impact on students’ learning strategies was explored. Raters were interviewed in focus groups or interviews to get a broader picture – especially with regard to fairness aspects.
Results: Children were asked during mini-interviews if they were satisfied that they could act as SP. Most of them were really happy (5-point Likert scale; 1 did not like it at all – 5 liked it very much; Mean = 4.6, SD = .70). In focus groups and interviews, raters’ views were recoded. They all agreed that the pediatric station was perceived as fair, feasible and realistic. Students well accepted the involvement of children SPs and that they perceived it as very realistic and as a good way to show their skills. 53% of the students expected a pediatric station where children SPs were involved. Of these 53%, 30.6% mentioned that this expectation had an effect on their learning.
Discussion:The goals of OSCE examination are to have comparable and reproducible clinical situations to judge not only knowledge but especially clinical feasibilities to handle the situation and last but not least the patient. Thus, including children in testing this setting in pediatrics seems mandatory and worth the significant additional workload in performing this examination.
Conclusions: Results show that the pediatric station was fair, feasible and realistic. From the view of all groups (students, raters, children), all participants were satisfied with the new approach of children SPs in pediatric OSCE stations. Children enjoyed participating in this setting, raters well accepted the process and evaluated the pediatric OSCE station as fair and feasible, and students perceived this station as fair and acceptable.
Take Home Message: Children are special patients and need special treatments – this should also be reflected in OSCE examinations.
References:
Darling JC, Bardgett RJM. Primary school children in a large-scale OSCE: Recipe for disaster or formula for success? Medical Teacher. 2013;35:858-861.
Cilliers FJ, Schuwirth LW, Adendorff HJ, Herman N, van der Vleuten CP (2010): The mechanism of impact of summative assessment on medical students’ learning. Adv Health Sci Educ Theory Pract. 2010 Dec;15(5):695-715.

Noemi Schaffner
Bern University of Applied Sciences, Switzerland; noemi.schaffner@bfh.ch
Background: In the “Bachelor of Science in Nursing” program at the Bern University of Applied Sciences (BFH), clinical exams, like Objective Structured Clinical Examinations (OSCE), are core to promote and ensure a high quality performance in clinical practice. Nursing students pass therefore three OSCEs during their studies. Normally an OSCE at the BFH consists of nine stations, which are conducted parallel on two floors with the goal to assess more students at the same time. Consequently for each station with the same scenario, two examiners and two standardized patients (SP) are needed, each of them responsible for an identical station on a different floor. Systematic measurement error can arise from this approach. This variance can be introduced by the performance of the SPs or the raters. For instance one examiner could tend to rate all students performances more stringent while the other tends to be more lenient. As a result students who are rated by the second rater get better grades than those rated by the first one. It is important to take measures to control the rater effect, starting by monitoring the rater differences.
Project Description: Different measures are taken to ensure the quality of the OSCE and to reduce possible, rater variance. Firstly, examiners and SPs who are responsible for the same OSCE station discuss and review each item on the checklist and ensure therefore standardization. Secondly, every station is statistically analyzed to detect rating differences between raters. By this means stations with statistically significant rater differences are identified. However with this analysis it isn’t possible to identify the problematic items. Therefore we, thirdly, analyze rating differences on each item of the “problematic” station.
Outcome: The results of the analysis are reported to the person responsible for the OSCE. This person informs all raters and SPs who are involved in the station with rating differences. Possible causes which lead to the differences are discussed (different understanding of the item, different performance of SP, ambiguous item wording) and measures are taken to eliminate them.
Challenges: Differences in OSCE outcomes can never only be explained by differences in raters or SPs. They can also simply be explained by differences in student’s performances. There is always a chance that all low-performing students are, by coincidence, assigned to the same floor. Nevertheless it is important to reduce differences due to this specific OSCE procedure.
Discussion: The analysis undertaken to ensure quality of the OSCE and to reduce rating differences are constructive. Raters in particular are eased by the knowledge of possible causes for the differences in their rating and are motivated to decrease discrepancies. Moreover this practice ensures that OSCE items are continually reviewed and improved.

Beatrice Buss
University of Applied Science Bern, Switzerland; beatrice.buss@bfh.ch
Background: The Objective Structured Examination (OSCE) measures the clinical skills in medicine. Checklists or global assessments criteria are used. Since 2006 OSCE, to measure clinical competence, is used by the bachelor physiotherapy course at the Bern University of Applied Sciences. The valuation is determined by global criteria – communication, therapeutic climate, technical skills and clinical decision-making ability. The purpose of this study was to determine if there is a difference in rating between therapeutic climate and communication.
Methods: This study investigated the ratings of the items communication and therapeutic climate. 50 students in the 1th Semester and 49 students in the 5th semester were analyzed. The item communication was rated by an examiner and the other item, therapeutic climate, by a standardized patient. Data were analyzed using the SPSS program (Version 2015).
Results: Statistically significant differences were not found in the ratings of the therapeutic climate and the communicative competence for physiotherapy students in the 1th Semester on 8 stations with a duration from eight minutes (p=0.16 to p=.956). In the 5th Semester there was no statistically significance on 3 Stations with a durance of twenty minutes (p=034 to p= 0.485), and 2 Stations from eight minutes.
Conclusion: In short stations we couldn’t find a difference between communication competence and therapeutic climate. The results were scattered among the single stations. It could be possible that in short stations the student is not able to establish a good relationship with standardized patients. However, in long stations the ratings by examiners were not statistically significant different to the short ones either. The rating of these items are independent and do not affect each other. Therefore the rating from standardized patients is important for student’s feedback.