Facilitating Aboriginal perinatal mental health information access with a retrieval-augmented LLM-based chatbot
Author Identifier (ORCID)
Wai Hang Kwok: https://orcid.org/0000-0002-1854-4300
Abstract
Aboriginal pregnant women and new mothers face an increased risk of mental health issues, often stemming from historical trauma, including violence and discrimination. These challenges could contribute to complex trauma and adverse perinatal outcomes, highlighting the need for culturally sensitive care. However, non-Aboriginal clinicians often face barriers due to limited cultural knowledge, exacerbated by other factors such as time constraints for training and reliance on one-time training. Large Language Models (LLMs)-based chatbots offer the potential to support self-directed learning and enhance clinicians' self-efficacy through interactive question and answer. However, LLMs also pose challenges, including hallucinatory responses, outdated knowledge, fictitious information, unverifiable references, and difficulty handling domain-specific queries. In this study, we aim to mitigate these challenges by developing a specialized chatbot for improving Aboriginal perinatal mental health question-answering. The chatbot integrates Retrieval-Augmented Generation (RAG) with a semantic search engine, enabling it to retrieve verified external knowledge and provide more accurate, contextually relevant responses without frequent retraining. We evaluate its performance against a baseline GPT-3.5-turbo model and compare LLMs integrated with different RAG techniques to assess improvements in accuracy and reliability.Clinical Relevance - This study shows the potential of the specialized RAG LLM-based chatbot to improve domain-specific, clinically relevant, and on-demand question-answering support for clinicians. By providing accurate, verified information through interactive responses, it may help bridge knowledge gaps, support self-directed learning, and complement existing training.
Keywords
Aboriginal perinatal mental health, large language models, retrieval-augmented generation, clinical decision support, cultural safety, healthcare chatbots
Document Type
Conference Proceeding
Date of Publication
1-1-2025
PubMed ID
41336738
Publication Title
Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
Publisher
IEEE
School
School of Nursing and Midwifery
Funders
Western Australian Future Health Research & Innovation Fund (IC2023-GAIA/18, IC2023-GAIA/23) / Google Academic Research Award
Copyright
subscription content
Comments
Utami, M. S. M., Kwok, W. H., Kotz, J., Walker, R., Wang, G., & Marriott, R. (2025). Facilitating Aboriginal perinatal mental health information access with a retrieval-augmented LLM-based chatbot. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 1–7). IEEE. https://doi.org/10.1109/EMBC58623.2025.11254329