the New England Section of the American Urological Association the New England Section of the American Urological Association
Search Meeting Site
Annual Meeting Home
Preliminary Program
Allied Health Program
Past & Future Meetings
 

Back to Annual Meeting Program


Comparison of Computer Generated Performance Metrics on the da Vinci Skills Simulator - Which Demonstrate the Most Construct Validity?
Kyle T. Finnegan, BS, Ryan P. Dorin, MD, Halil Kiziloz, MD, Steven J. Shichman, MD.
Hartford Hospital, Hartford, CT, USA.

BACKGROUND:
Previous studies have demonstrated construct validity of the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA) for differentiating novice, intermediate, and expert robotic surgeons in overall score and time to completion of simulator tasks. Our purpose is to determine the relative construct validity of specific performance metrics within the Skills Simulator scoring system, thereby elucidating which metrics may be particularly useful to incorporate in a robotic surgery training curriculum.
METHODS: Thirty-nine surgeons (18 group I [0-20 robotic cases]; 8 group II [21-150 robotic cases]; and 13 group III [>150 robotic cases]) were enrolled from September 2010 to December 2010. Participants completed all 24 virtual-reality exercises available on the da Vinci Skills Simulator at the time. Data on 12 performance metrics was collected by the software. The ability of each of the 12 metrics to differentiate between surgeon experience groups was analyzed using univariate general linear model and analysis of variance tests.
RESULTS: Of the 12 metrics, 7 were applicable to all 24 exercises, with 5 metrics (i.e. misapplied energy time) only being applicable to select exercises. Of the 7 universally applied metrics, scores on ‘Time to Complete,’ ‘Economy of Motion,’ and ‘Work Space Range’ were significantly different between Group I and Group II surgeons across the greatest number of exercises (10 or more exercises out of the 24 completed). Only ‘Time to Complete’ demonstrated significant differentiation between Group II and Group III surgeons across a variety of exercises (Table 1).
CONCLUSIONS: Within the da Vinci Simulator exercise library, metrics relating to efficient use of the console workspace were most effective in differentiating novice surgeons from intermediate surgeons, while the speed metric was most effective in differentiating intermediate from expert surgeons. A simulator curriculum focusing on economy of motion and efficient clutching may be especially effective in training the novice robotic surgeon.
Table 1 - Significant differences in exerces on da Vinci Skills Simulator between Groups
Time to CompleteEconomy of MotionWork Space RangeInstrument CollisionsCritical ErrorsExcessive ForceInstument out of view
Group I vs. Group II11/24(46%)12/24(50%)10/24(42%)9/24(38%)6/24(25%)4/24(17%)1/24(4%)
Group II vs. Group III16/24(67%)3/24(13%)1/24(4%)1/24(4%)2/24(8%)1/24(4%)1/24(4%)


Back to Annual Meeting Program

 


© 2022 New England Section of the American Urological Association. All Rights Reserved. Privacy Policy.