Laparoscopic skills training of surgical residents : a comparison of two proficiency-based independent approaches

Aim: Current financial and work hour constraints make proctored on-site laparoscopic simulation training challenging. An independent learning approach utilizing proficiency-based training is a potential solution. The purpose of this study was to determine if an independent approach using a portable, laparoscopic training device within one’s home environment could effectively train novices in laparoscopic procedural skills. Methods: After baseline testing, laparoscopic novices (n = 16) were randomized to one of two study groups. The on-site group (n = 7) received unlimited access to the workplace laparoscopic trainers and the home group (n = 9) received portable laparoscopic trainers for home. Both groups underwent self-directed, proficiency-based training for three months then were retested. Results were compared with parametric and non-parametric statistical tests. Results: Baseline characteristics were similar between groups. The practice rate (56%) and practice time (range, 0.18 to 2.6 h) were poor in both groups during the training period. At post-test, the number of participants who demonstrated an improvement (86% on-site, 78% home) on the peg task was not different between groups. The successful completion of the suturing task post-test had significantly improved compared with pre-test in both groups (71% vs. 29% on-site; 44% vs. 22% home, P < 0.001). Although the majority of participants reported it was difficult to practice on a regular basis (86% on-site, 89% home), 56% of the home group participants agreed that the at-home trainer was a helpful teaching modality. Conclusion: Learning of laparoscopic skills by novice trainees can be augmented by an independent learning approach using either home or on-site laparoscopic trainers. Although over half the candidates found it was useful to have the training device at home, none of the participants practiced more than an hour or two in the three month training period. Thus, the solution to conducting training does not lie in merely providing home training, but rather to understand the work-related stressors and reconfigure jobs.


INTRODUCTION
To maximize the benefit of time in the operating room, laparoscopic skills training outside the operating room has become the gold standard for educating surgical residents.It is well documented that residents benefit from this form of training and can lead to improved operative performance.In fact, training is an integral and essential component of a trainee's job.However, there is no consensus regarding the optimal method or learning environment for teaching laparoscopic skills in order to maximize trainees' education while maintaining an equitable and sustainable work-life balance.[3] However, this approach is not without disadvantages.[9] In fact, duty-hour limitations have led to a significant re-evaluation of the traditional surgical education paradigm, with emphasis on increased efficiency of educational efforts.Accordingly, a disadvantage of these on-site training facilities is that they are often not readily accessible to the resident.Unless dedicated training time is made mandatory for residents, few residents independently take advantage of these costly facilities during their already busy week. [10]If educators seek to facilitate laparoscopic learning, they should provide the trainees with the opportunities for extracurricular practice.Finally, there is also an economical aspect to consider as the use of surgical educators can be costly, time consuming, and difficult to achieve for smaller institutions.
An independent learning approach where trainees are given self-study resources (i.e.video/didactic tutorials) as well as unlimited access to the on-site training facility and encouraged to practice at their own pace is one alternative strategy. [6]The advantages of such self-directed training method are reduced instructor time and simplified scheduling.[13] However, in the United States, the training time on-site within the workplace environment is counted toward the already restricted resident duty hours.A hybrid approach that employs an independent learning approach using at-home, portable laparoscopic trainer outside the time constraints of the hospital environment, has not been fully investigated.As medical educators and mentors, the best methods of teaching laparoscopic skills and the most efficacious learning environment where trainees feel motivated and allocate a high priority to practice needs to be the standard for resident education.Clearly, laparoscopic training should not be considered after exhaustion from work-related activities.In fact, there is evidence to suggest that motor training during periods of exhaustion can deteriorate and may create an environment that promotes poor technique. [14]This is precisely why home training may be beneficial.It would be portable, accessible, convenient, flexible and inexpensive once initiated.Indeed, trainees could practice the laparoscopic skills at their preferred time, for example when are well-rested and away from work stressors, and not at a time that fits others' schedules.Therefore, we hypothesized that an athome independent, proficiency-based laparoscopy training method using a low-cost device will improve the laparoscopic skills of novice trainees and that this training method-environment combination would be equivalent to traditional, independent hospital-based, laparoscopic training.The primary endpoint of the study was to compare trainee performance before and after at home practice training.

Participants
Novice laparoscopists [First and Second Post Graduate Year (PGY-1 and 2) general surgery residents, (n = 8) and medical students (n = 9), at any year of training] were recruited into the study through word of mouth at Tulane University Medical Center.Prior to beginning training, all participants completed a questionnaire assessing demographics and previous exposure to laparoscopy.The Tulane University Institutional Review Board approved this study, and all subjects gave informed consent prior to participation.

Apparatus
The Minimally Invasive Surgical Trainer (Joystick SimScope™; 3D Med, Franklin, Ohio) was used in this study for the home training device.This training device is a self-contained, lightweight (15 lbs), and portable box equipped with a camera and 10 inch LCD color monitor, which offered good visual resolution, and optics with all the close-up and rotation options.The trainer is also equipped with seven ports fitted with grommets designed to hold an instrument or a trocar.Tasks were carried out with the monitor at eye level and the laparoscopic instruments at a standard surgical height between the monitor and the participant.
The on-site "standard" conventional trainer (Karl Storz Endoscopy, Culver City, CA) used in this study included a 15 inch video monitor (Sony Corporation, New York, NY), Xenon-nova light source, Telecam SL camera system, Hopkins II laparoscope and a Plexiglas box trainer.Tasks were carried out using a 0° Strorz 10-mm laparoscope connected to a light source and with the images directed to the Sony television monitor.

Training and testing protocol
After an orientation and viewing a introductory video on the "Fundamentals of Laparoscopic Surgery" (FLS) peg board transfer and intra-corporal suturing and knot tying tasks, [15] all trainees completed a baseline assessment (pre-test) on these tasks using the onsite "standard" video-trainer.The peg transfer and an FLS-type video-trainer laparoscopic suturing model were used to assess their baseline skills.Performance scores were calculated and recorded for each peg transfer attempt using time (s) and for each laparoscopic suturing and knot tying attempt using the previously published formula: 600 -[time (s) + 10* accuracy error + 10* security error]. [16]If there was failure to complete the later task (i.e.tie a functional knot) within the time limit (10 min), the task was terminated and the participant was given a score of 0. Baseline performance was defined as the mean score of the first three repetitions at the beginning of training (in the absence live, proctored instruction).Subjects were then ranked according to the sum of the overall scores for the three attempts, stratified into blocks of two and randomized into two groups.On-site group received unlimited 24-h access to the on-site skills laboratory for independent practice.Home group received a selfcontained, portable laparoscopic -minimally invasive training system box (Joystick SimScope™; 3D Med, Franklin, Ohio).
Both groups were then allowed to self-direct their training for a three months period of time.During this independent training period, both groups were given access and allowed unlimited viewing of the didactic tutorials on the two tasks.The peg transfer model, an FLS-type video-trainer laparoscopic suturing model, and 6-inch pre-cut 3-0 silk sutures were provided to each participant.All participants were given previously established task specific proficiency levels for the peg transfer (48 s) [17] and the laparoscopic suturing model (score 512) [16] at the start of the training period to guide practice.To further foster goal-directed learning, all participants were encouraged to train as long as they needed in their spare time until they reached the predefined proficiency criterion.Both groups were given a journal to record number of practice days, and time spent practicing on each task over the 3-month independent training period.After training completion, all participants underwent repeated evaluation (posttest) on the same initial two laparoscopic tasks using the on-site "standard" video-trainer.

Questionnaire
Each participant completed a questionnaire on the educational experience, at the completion of the study investigating the perceived benefit of the training method.One set of issues concerned the opportunity for practice during the study period while another concerned the usefulness of the home training device in term of the learning of surgical skills.Participants in the home group were asked to evaluate using 10-point Likert scale (1-10) their satisfaction regarding the home training with higher numbers being more positive responses.

Statistics
Data are expressed as means ± standard error of the mean.Comparisons of the pre-training and the posttraining continuous variables for each domain within groups were performed using a Wilcoxon matched pairs test.Comparisons of continuous variables between groups were conducted by using an unpaired two-tailed t-test.Categorical variables were compared using Fisher's exact or chi square test.Computer software (GraphPad Instat software, San Diego CA) was used for all statistical analyses.A P value of less than 0.05 was considered significant.

Study population
Seventeen subjects were enrolled, but one subject (medical student) dropped out secondary medical reasons during the training period, and this individual was not included in the final study analysis.Therefore, the subsequent analysis was per protocol.The mean age of the study population was 31.0 ± 1.5 years (range 24-47 years).Nine subjects were female (52.9%) and 15 were right-hand dominant (88.2%).There were no significant differences in age, gender, or self-reported laparoscopic experience, between groups.Moreover, trainee baseline simulator performances for the two groups were equal (P > 0.05).

Training period
As a group, only 56 % (n = 9/16) of the participants actually practiced the laparoscopic tasks during the training period (n = 4 in on-site group, n = 5 in home group).Of the participants that practiced (50% of medical students, 40% of PGY-1, and 33% of PGY-2 residents) only one subject (on-site group; medical student) practiced on a regular basis with a total 129 training time of over 29 h.For the other participants who practiced the average total recorded training time was just 1.0 h (range 0.18-2.6h), over 5 different days (range 1-15 days) during the entire 3-month training period.
The task-specific practice was as follows: for the pegboard task, only one subject (a medical student) in the on-site training group actually trained on this task during the training period.By comparison, 4 out of 9 (44.4%)subject in the home group practice this task.For the suturing task, nine people (56%) practiced during the 3-month training period (4 in the on-site group with total training time 5.0 ± 4.3 h; 5 in the home group with total training time 1.0 ± 0.8 h).

Post-test skills assessment performance
Since the number of participants in each group that actually practiced during the study period was small, this situation precluded any meaningful statistical breakdown for this subgroup.Therefore, for the posttest analysis all participants (i.e.those who did and did not train at all) were included.Interestingly, most participants achieved improvements regardless of which group they were initially assigned for training or the amount of practice they recorded.For the pegboard task, post-training times for the participants in the onsite group improved on average 65 s (37%) compared to pretest scores (177.7 ± 23.8 s pre vs. 112.2± 9.3 s post, P = 0.047).By comparison, the home group improved on average 41 s (22%), compared to pretest scores (183 ± 21.5 vs. 142.9 ± 16.6 s, P = 0.039).The most marked improvement was noted with one participant improving the pegboard time by 196 s (onsite group).Interestingly, the number of participants who improved on the pegboard task (86% vs. 78%) and the average time to task completion after training was not statistically significantly between the home and on-site groups, respectively (P = 0.47).Importantly, despite these improvements no participants, in either group, achieved proficiency at re-testing for the pegboard task.
As expected, the more complex task (suturing) provided the greater training challenge, however an improvement was noted in both groups.The suturing task completion rate for both groups had significantly improved after the training period (71% vs. 29% onsite group; P < 0.001 and 44% vs. 22% home group; P < 0.001).Finally, at re-test, the on-site group score improved by an average of 113 points (114.9 ± 74.6 pre vs. 228.4± 83.6 post, P > 0.05) compared to 39 points for the group trained at home on the portable device (80 ± 53.5 pre vs. 118.7 ± 60.2 post, P > 0.05 Wilcoxon matched pairs test).Although the magnitude of improvement on the suturing task was significantly greater for the on-site trained group on both forms of assessment, no statistically significant difference could be demonstrated (P = 0.54).Only one participant from the on-site group achieved proficiency on the suturing task.However, this individual practiced significantly more than all other participants (18 h) during the independent training period.

End-of-study questionnaire
In total, 93% reported no or minimal exposure to laparoscopic surgery during the study period.The ease of use of the home training device was evaluated using a 10-point Likert scale with anchored end points; 1 being easy and 10 being difficult.Many respondents felt that the home box trainer was easy to use with a median score of 2. According to half of participants, the study experience was beneficial to their laparoscopic skills education.Importantly, regardless of the training location, a majority of each group stated they had difficulty practicing regularly (86% in on-site group, 89% in home group).The 44% who do not practice cited the following reasons: lack of time (57%), away rotations (29%), and the remaining 14% cited various other reasons.Of the 56% who did practice still cited a lack of time (56%) as the major reason they did not practice more often.

DISCUSSION
Laparoscopic training is an integral and essential component of a surgical trainee's job.However, within a changing surgical environment alternative methods for laparoscopic training must be sought for training which incorporate opportunities to practice.In this un-blinded, randomized study, two proficiencybased independent approaches were employed to teach laparoscopic skills to beginners, with one tactic employing standard on-site physical box trainers at the workplace and the other relying on a similar device used in the trainee's home, outside the stress of the work environment.Both methods allowed trainees to practice their laparoscopic skills at their own pace.In the beginning, pre-training skills were homogenous with minimal baseline experience in the two groups.In the end, novice trainees showed improvement in their laparoscopic skills using our self-directed, proficiency-based home training program.However, at most, it was comparable to our on-site program in terms of feasibility and rates of participation.Several studies have shown that learning can be facilitated if learners are able to self-direct their own training experience [11][12][13] while other studies have shown that low cost, portable training device can improve laparoscopic skills. [18,19]However, the ability of a self-directed laparoscopic training experience using lowcost portable training devices to improve laparoscopic procedural skills with the home environment has not been investigated.We used a proficiency-based, self-directed training approach both within the home and workplace environment.Both groups were instructed to practice at their discretion to achieve expert proficiency on each task with no ramifications if they did not succeed.We believed this would be successful based on the observation that surgical trainees and those interested in surgery are highly motivated to learn the required skills and invest the necessary time.In addition, there seems to be natural selection within the surgical population itself.Trainee motivation is essential for learning because it promotes deliberate practice and persistent efforts to improve objective performance. [20]Without motivated learners, any educational efforts will have limited achievements and the skills laboratories will be attended infrequently.At the end of the day, our method was met with mixed results.First of all, the self-directed training approach did enhance the novices' laparoscopic skills in both groups.But, we also observed that few subjects attended the onsite skill laboratory during their independent training period.More surprising, was the observation that when subjects were given the training device to use at home, they still did not find the time to frequently train independently.Interestingly, on average, the home trainees found only about 1-2 h to practice within a three-month period of time and 44% never practiced once during this period.These results demonstrated that one of the most important factors in getting trainees to practice and hone their laparoscopic skills was providing them with enough free time during their work week not changing their learning environment.This ultimately disproved our original hypothesis that moving practice opportunities out of the busy and stressful clinical environment to "free time" would be beneficial.Thus the hard, but unavoidable truth appears to be that there are barriers to practice in both settings that need to be better understood.
The fact that the groups demonstrated improvement despite low recorded practice rates during the training period may indicate that this is due to random effects, rather than due to the amount of deliberate practice.[23][24][25] In our study, all subjects most likely received learning from the videobased instruction and during the practice for the period of the pre-test scoring.Indeed, video-based instruction has been shown to be efficacious in the development of laparoscopic skills. [26,27]It has been recently demonstrated that under prescribed practice conditions, video based instruction is equally effective as faculty instruction in teaching basic surgical skills to novice trainees. [23,24,28]pervised practice-feedback is believed to be another foundation of effective learning.Feedback and formative assessment refer to information about performance that is intended to guide learning.The purpose of giving feedback is to encourage learners to think about their performance and how they might improve.However, feedback in practice is often vague and evaluative (e.g."good suturing"). [29]While direct intensive practice feedback was not given in our study, the use of pre-set task-specific proficiency criteria to guide practice was utilized.This has been shown to give the informative feedback and opportunity for error correction vital for deliberative practice and can improve laparoscopic training. [6]his may be another reason for the observed skill improvements in both groups.
Our study has several limitations and must be viewed cautiously as they may not apply for other simulators or other subjects with different motivation, interests, and backgrounds.If all of our subjects were surgical residents that were required to achieve proficiency prior to being allowed into the operating room, we believe that nearly 100% would attain the predefined proficiency levels as opposed to the 6% noted in the current study. [30]Another limitation of our study was that we assumed that the participants knew how to self-direct their learning.Perhaps with more oversight, including feedback and good practice reinforcement, the path to greater task improvements would have been identified.Another limitation was that the number of participants in our study was small.Interestingly, for both tasks the greater improvements were with the on-site training -although statistical significance between the two groups could not be demonstrated this may be a function of small numbers.A larger cohort might result in statistically significant differences between the two groups.A further limitation of this study is it did not attempt to correlate improvement in task performance with improved performance in live human operations.Despite these limitations, this study has provided the foundation for additional assessment of the home trainer as a means of improving operative performance.
Essentially, our investigation is a feasibility study evaluating if home training is viable, however it was not performed to determine if programs can "avoid obligations" and "negate" work hours regulations.Our study found that novice individuals trained on either the home or the on-site training device were able to improve their laparoscopic skills objectively and subjectively using an independent proficiency-based training method.However, none of the participants practiced for more than 1 or 2 h in the 3-month training period.Therefore, we feel that the only valid conclusion that can be drawn is that when overworked and exhausted trainees who are expected to train on their own time, do not regularly engage in training exercises, regardless of whether these are provided at home or in the workplace.The solution to conducting training in a stressful work environment may not be to simply "suggest" that trainees practice at home, but rather to understand the work-related stressors and reconfigure jobs, and perhaps even to increase staffing, to minimize stress exposure.