To develop and evaluate a mobile phone-based skills assessment tool that measures procedural competency of urology residents learning to perform a common, non-robotic urology procedure as a means of tracking current skillset and improvement over time.
The assessment tool was a Qualtrics survey accessed via a smartphone link that breaks down a vasectomy into 6 critical steps. Level of competency was measured on a scale of “1-novice” to “5-expert.” Nine residents from PGY-1 to PGY-5 were evaluated by one instructor after completing a vasectomy (86 single-side cases recorded over a 6-month period). We compared individual trainees to each other, analyzed performance (improvement) over time, and evaluated competency against cohort and program averages.
As an example, a single resident (“Resident 2,” N=11 cases) was compared to cohort (PGY, M=7.5/resident) and program (all residents, M=7.4/resident). Results indicate similar skillfulness across Step 1 (puncturing & isolation of vas and hand positioning; p> 0.1), but marginally lower competency on Step 2 (opening of vasal sheath to expose/isolate vas; vs. cohort: p=0.076, vs. residents: p=0.082). Significantly lower competency on Steps 3-6 (all p<0.04) suggests targeted teaching could improve cautery technique, fascial interposition, hemostasis, and positioning of stumps.
Our mobile-based skills assessment is a low cost, novel, and efficient assessment that would support current ACGME goals to increase competency-based residency training. This tool is easily created and accessed, provides real-time feedback to learners, and can be used for individual and group assessment at a single timepoint or longitudinally.

Copyright © 2023 Elsevier Inc. All rights reserved.