Logo image
Designing a safe generative AI-powered chat assistance system for mental health using a bounded generative framework
Journal article   Open access   Peer reviewed   PUBLISHED

Designing a safe generative AI-powered chat assistance system for mental health using a bounded generative framework

Udochukwu John-bright Okike, Amarachi Eche, Chimeremeze Victor Amaechi, Pradeep Hewage, Anchal Garg and Adel Houten
Discover Artificial Intelligence, Vol.6(1), 1584
23/10/2026

Abstract

Therapeutic AI large language models Emotional Freedom Techniques safety architecture human-AI interaction clinical decision support system
Purpose: Mental health services worldwide face demand that consistently exceeds available capacity, leaving many people without timely access to support. Generative AI chat assistants offer a scalable way to help close this gap, but only if their conversational flexibility can be - reconciled with the protocol fidelity, crisis safety, and auditability that safety-critical therapeutic use demands. This paper proposes the Bounded Generative Framework (BGF) as an architectural pattern for deploying large language models (LLMs) in such contexts, using an Emotional Freedom Techniques (EFT) chat intervention as a case study. It addresses this bounded generativity problem: retaining empathic, adaptive responses while enforcing protocol fidelity, crisis safety, and auditable behaviour. Methodology: We implemented the BGF in Tapaway, a web-based EFT application that integrates a two-layer pipeline, a 10-state therapeutic protocol controller plus a terminal crisis state, and structured tool calls for JSON-based directives. We compared three LLMs (GPT-4o, Claude 3 Haiku, and Gemini 2.5 Flash). Technical performance was evaluated on a 29-case synthetic benchmark spanning standard, edge, adversarial, crisis, and protocol-compliance scenarios. Clinical quality was explored using authentic EFT sessions. LLM security was examined via OWASP-aligned adversarial tests targeting prompt injection, unsafe output handling, and data leakage. Results: Across synthetic cases, models achieved high JSON parse success and directive validity, with GPT-4o yielding the strongest protocol fidelity and crisis-intent performance at the cost of higher latency. In application, paired sessions in this uncontrolled convenience sample of university students showed large within-session reductions in self-reported distress (Cohen's d = 1.85), which should be interpreted as a descriptive feasibility signal rather than evidence of efficacy. Crisis detection yielded no false negatives in this sample (sensitivity 100%, 95% CI 69.2 – 100% across N=10 combined crisis-positive cases), though these preliminary findings require validation in adequately powered studies, and security testing indicated partial robustness to common LLM-specific risks. Conclusion: The BGF shows that safety-bounded, protocol-controlled generative interventions are feasible for EFT chat delivery, shifting assurance from model-only alignment towards application-level constraints. While promising as a design pattern for therapeutic AI, the framework does not establish clinical equivalence to human-delivered therapy; controlled trials are required to assess comparative and durable outcomes.
pdf
Designing a safe generative AI-powered chat assistance system for mental health using a bounded generative framework2.23 MBDownloadView
Published (Version of record) Open Access CC BY-NC-ND V4.0  — This license enables reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only if attribution is given to the creator.
url
Link to Published VersionView
Published (Version of record) Open Access Open CC BY-NC-ND V4.0  — This license enables reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only if attribution is given to the creator.

Metrics

18 Record Views

Details

Logo image

Usage Policy