Publication Date

Spring 2026

Degree Type

Thesis

Degree Name

Master of Science (MS)

Department

Applied Data Science

Advisor

Mohammad Masum; Guannan Liu; Sayma Akhter

Abstract

This research presents a two-stage pipeline for user-level suicide risk detection from Reddit: first, inference-only prompting with summarization; second, fine-tuned encoder classification with explainability and expert validation. The data are user-level: each of the 500 C-SSRS Reddit items is one user’s chronologically concatenated posts and comments (a user timeline), annotated by psychiatrists. Stage one: six prompting strategies zero-shot, few-shot, chain-of-thought, tree-of-thought, least-to-most, and self-consistency are evaluated across six LLMs on multi-class and binary formulations; simple zero-shot achieves the highest balanced accuracy (0.53 multi-class). Error analysis shows longer inputs associate with misclassification (p = 0.002); domain-specific summarization (timelines >2,000 tokens) reduces length by 44%, preserves semantic fidelity (BERTScore F1 = 0.85), improves smaller models (up to 10%), and yields roughly 32% cost savings. Classical baselines (best BA = 0.31) underscore LLM advantages. Stage two: the mental-longformer-base-4096 encoder is fine-tuned on summarized C-SSRS and eRisk 2025 timelines for binary (risk vs. no risk) classification. A 7-configuration ablation over C-SSRS, eRisk, and ReDSM5 motivates the C-SSRS+eRisk main model (mean BA 0.89±0.01; best 0.90), with zero-shot transfer to external datasets (e.g., Twitter BA 0.85, ROC-AUC 0.92). Integrated Gradients and attention visualizations provide token-level explanations; three independent clinicians rate clinical-style summaries of model rationales (84% majority agreement). Multi-class addresses granularity; binary and fine-tuning address robustness. Deployment requires large-scale validation, fairness assessment, and human oversight.

Download

Share

COinS