Back to 2025 Abstracts
DOES ARTIFICIAL INTELLIGENCE ADHERE TO AUA GUIDELINES IN THE MANAGEMENT OF PATIENTS WITH URETERAL AND RENAL STONES?
Sydney Look-Why, BA1, Daniel Givner, MD
2, Bouyon Xiao, BS
1, David Wang, MD
2.
1Chobanian and Avedisian Boston University School of Medicine, Boston, MA, USA,
2Boston Medical Center, Boston, MA, USA.
BACKGROUND:Artificial intelligence (AI), particularly large language models (LLM) like ChatGPT, has increased in popularity. AI use, however, is still in its infancy. In urology, AUA Guidelines have been published for numerous conditions but might not be familiar to non-urologic providers and patients. Therefore, the objective of this study is to assess the accuracy ChatGPT offers relative to the AUA guidelines for medical and surgical management of patients with nephrolithiasis, when to consult a urologist, and how to counsel patients with nephrolithiasis.
METHODS:The AUA offers 83 guidelines for management of nephrolithiasis, of which 32 guidelines were utilized. The guidelines were then reorganized into clinical vignettes from the perspective of a non-urologic physician and inputted into ChatGPT. The answers were then compared by multiple reviewers to the AUA guidelines. Data was analyzed through a binary system such that, complete alignment of AUA guidelines and ChatGPT answers scored a 1.0, whereas ChatGPT answers that did not reflect AUA guidelines scored a 0.0. Correct answers were summated and divided by total number of guidelines to yield a percent correct score.
RESULTS:Of the 32 guidelines utilized, 12 guidelines reflected the medical management guidelines, while 20 guidelines reflected the surgical management guidelines. 31 of the 32 ChatGPT answers were found to align with the AUA guidelines (31/32 = 96.8%). All 12 of the ChatGPT answers to clinical vignettes targeting AUA medical guidelines were found to align (12/12 = 100%), whereas 19 of the 20 ChatGPT answers to vignettes targeting AUA surgical guidelines were found to align completely (19/20 = 95%).
CONCLUSIONS:Our study demonstrated that the use of AI, was extremely accurate in comparison to the AUA guidelines related to nephrolithiasis. This pilot study demonstrated AI’s significant potential in guiding non-urologic physicians and patients in management of nephrolithiasis, and necessity for further studies evaluating this tool.
Back to 2025 Abstracts
