Skip to main content
2025 ASCRS Annual Scientific Meeting Main banner
Final Program


Visit ASCRS U
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Presentation Icons
CME
CME
Invitation/Ticketed Event
Invitation/Ticketed Event
Invited Lectureship
Invited Lectureship
Award Recipient
Award Recipient
Self-Assessment
Self-Assessment
Social
Social
Industry
Industry
Back

Session: Abstract Session: General Surgery Forum

GS5 - Enhancing Surgical Patient Education: Evaluating the Accuracy and Readability of AI-Translated Materials Across Multiple Languages

Sunday, May 11, 2025
2:52 PM – 2:56 PM PT
Location: Ballroom 6B, Upper Level

    Abstract Podium Presenter(s)

  • PH

    Phillip W. Hwang, MD

    Surgery Resident
    New York Presbyterian Queens
    Port Washington, NY, United States

    All of the relevant financial relationships listed below have been mitigated.
    This individual has no financial relationships with ineligible companies.

    • Author(s)

  • Julie Hong, MD

    Resident
    NewYork Presbyterian - Queens
    Flushing, NY, United States

    All of the relevant financial relationships listed below have been mitigated.
    This individual has no financial relationships with ineligible companies.

  • WH

    Wan Huang, n/a

    Resident
    New York Presbyterian Queens
    New York, New York, United States

    Disclosure information not submitted.

  • OF

    Omar Font-Torres, n/a

    Resident
    New York Presbyterian Queens
    new york, New York, United States

    Disclosure information not submitted.

    • Senior Author(s)

  • SC

    Steven Y. Chao, MD FACS

    Assistant Professor of Clinical Surgery
    Weill Cornell Medicine
    Flushing, New York, United States

    All of the relevant financial relationships listed below have been mitigated.
    This individual has no financial relationships with ineligible companies.

    • Submitting Author(s)

  • PH

    Phillip W. Hwang, MD

    Surgery Resident
    New York Presbyterian Queens
    Port Washington, NY, United States

    All of the relevant financial relationships listed below have been mitigated.
    This individual has no financial relationships with ineligible companies.

All of the relevant financial relationships listed below have been mitigated.
Phillip W. Hwang, MD: This individual has no financial relationships with ineligible companies.
Julie Hong, MD: This individual has no financial relationships with ineligible companies.
Steven Y. Chao, MD FACS: This individual has no financial relationships with ineligible companies.
Image or Table (Do Not add text here):

Purpose/Background: The National Institutes of Health recommend that patient education materials (PEMs) be accessible, ideally at a sixth-grade reading level. However, many Americans have limited English proficiency, complicating access to PEMs. Large language models (LLMs) offer promising solutions for efficiently translating PEMs into multiple languages. This study evaluates whether artificial intelligence (AI) can reliably create ready-to-use PEMs in Spanish, Chinese, and Korean.

Methods/Interventions: We used several LLMs, including OpenAI's ChatGPT, Google's Gemini, Anthropic's Claude, and Meta’s Llama 3, to translate sixth-grade-level PEMs for common surgical conditions into Spanish, Mandarin, and Korean. Medical providers fluent in both English and the target languages evaluated the accuracy and grammar of these translations. Readability was assessed using language-specific tools: Flesch-Kincaid for English, Fernández Huerta for Spanish, Chinese Readability Index for Mandarin, and Natmal for Korean.

Results/Outcomes: AI-generated translations demonstrated high accuracy and grammatical correctness across all languages. A total of 11 fluent medical providers (4 Spanish-speaking, 3 Korean-speaking, and 4 Mandarin-speaking) agreed that AI-translated PEMs were comparable in accuracy to manually translated PEMs. Readability assessments showed that AI-translated texts generally met recommended complexity levels. AI-generated Spanish translations were consistently at an eighth-grade level. Korean translations ranged from sixth to ninth-grade level. Mandarin translations ranged from seventh to eighth grade. Human translations were between sixth to ninth grade for Korean and seventh to eighth grade for both Spanish and Mandarin.

Conclusion/Discussion: AI-driven translation of PEMs effectively addresses language barriers in healthcare communication by providing accurate and grammatically correct translations across multiple languages. The study demonstrates that AI translations generally meet or exceed recommended readability levels, enhancing patient comprehension and engagement. By bridging health literacy gaps, these tools have the potential to significantly improve health outcomes. Further research is needed to explore AI translation efficacy in diverse medical contexts and to refine these technologies for even greater accessibility and precision. Integrating AI into patient education represents a promising advancement in leveraging technology to enhance healthcare communication.