Back to 2026 Abstracts
Clinical Utility of Artificial Intelligence and ChatGPT in Adherence to American Urologic Association Guidelines Across Urologic Conditions
Joseph Liao, BS1, Karen Li, BS1, Jane Choe, BS1, Sydney Look-Why, BA1, Daniel Givner, MD2, David Wang, MD2.
1Chobanian and Avedisian Boston University School of Medicine, Boston, MA, USA, 2Boston Medical Center, Boston, MA, USA.
Introduction:The American Urological Association (AUA) Guidelines provide evidence-based recommendations for the diagnosis and management of urologic conditions, yet adherence remains inconsistent among non-urologic clinicians and patients. Artificial intelligence (AI), particularly large language models (LLMs) such as ChatGPT, may help bridge this gap by rapidly generating and communicating guideline-based information easily. This study evaluates the potential of AI to facilitate guideline-concordant care by assessing ChatGPT's ability to generate clinical recommendations consistent with AUA Guidelines across a broad range of urologic diseases.
Methods:Based on the AUA's 32 guidelines covering most urologic conditions with 1097 recommendations, 1097 clinical vignettes were created from the perspective of a patient or non-urologic provider, and inputted into ChatGPT 5. Responses from ChatGPT were compared to official guideline recommendations and concordance was recorded via multiple reviewers. Concordance was scored binary: 1.0 for complete alignment, 0.0 for non-alignment. Percent concordance was calculated as total correct responses divided by total guidelines tested.
Results:Of 1097 total guideline recommendations assessed, ChatGPT responses were in concordance for 983 of the recommendations, giving a total concordance score of 90.1%. There were mild differences in concordance accuracy among guidelines, with a standard deviation of 5.34%. Guideline recommendations for “Microhematuria” had the lowest concordance at 75.9% and both “Cryptorchidism” and “Bladder Cancer: Non-metastatic Muscle Invasive” had the highest concordance at 100%.
Conclusion:ChatGPT demonstrates high concordance with AUA guidelines across multiple urologic conditions, suggesting its potential as a clinical decision-support tool for non-urologic providers and patients. AI may enhance adherence to evidence-based care, improve patient counseling, and support timely specialist referral. Further studies are needed to evaluate real-world integration, coverage across all guidelines, and impact on clinical outcomes.Table 1. Congruence of ChatGPT and AUA Guideline Recommendations.
Back to 2026 Abstracts
