Hoboken, New Jersey
Hoboken, New Jersey
Hoboken, New Jersey
Far Eastern Memorial Hospital
National Yang Ming Chiao Tung University
Hoboken, New Jersey
Yi-Rou Ji
National Yang Ming Chiao Tung University
Hoboken, New Jersey
Alzheimer’s disease (AD) has become a prevalent neurodegenerative disease worldwide. Traditional diagnosis still relies heavily on medical imaging and clinical assessment by physicians, which is often time-consuming and resource-intensive in terms of both human expertise and healthcare resources. In recent years, large language models (LLMs) have been increasingly applied to the medical field using electronic health records (EHRs), yet their application in Alzheimer’s disease assessment remains limited, particularly given that AD involves complex multifactorial etiologies that are difficult to observe directly through imaging modalities. In this work, we propose leveraging LLMs to perform Chain-of-Thought (CoT) reasoning on patients’ clinical EHRs. Unlike direct fine-tuning of LLMs on EHR data for AD classification, our approach utilizes LLM-generated CoT reasoning paths to provide the model with explicit diagnostic rationale for AD assessment, followed by structured CoT-based predictions. This pipeline not only enhances the model’s ability to diagnose intrinsically complex factors but also improves the interpretability of the prediction process across different stages of AD progression. Experimental results demonstrate that the proposed CoT-based diagnostic framework significantly enhances stability and diagnostic performance across multiple CDR grading tasks, achieving up to a 15% improvement in F1 score compared to the zero-shot baseline method.
Alzheimer’s disease (AD) is the most prevalent neurodegenerative disorder worldwide, posing a growing challenge to healthcare systems and patients’ quality of life. The rising global prevalence of AD creates an urgent need for accurate, scalable, and resource-efficient diagnostic tools. Traditionally, standard diagnostic protocols remain heavily reliant on a combination of time-consuming and costly methods, including advanced medical imaging modalities such as PET and structural MRI, alongside clinical assessments by specialized physicians [
In recent years, the integration of large language models (LLMs) with EHR has transformed predictive analytics in medicine. LLMs are increasingly applied to diverse tasks ranging from clinical document summarization to patient outcome prediction. However, their direct application to inherently multifactorial complex diseases like AD remains constrained. Despite advances in LLMs for understanding medical texts, their application in diagnostic decision-making remains somewhat limited. Traditional LLM fine-tuning often produces black-box classifiers with limited interpretability and traceability, constraining their clinical adoption [
To bridge this critical gap in explainability and complex reasoning, we propose a novel diagnostic workflow leveraging Chain-of-Thought (CoT) reasoning within LLMs for AD assessment based on comprehensive clinical EHR data [
To address the current lack of interpretability and reliability in language models for Alzheimer’s disease (AD) diagnosis, this study primarily explores the following three key questions:
1. Will LLM reliably identify subtle CDR grading differences from unstructured EHRs?
2. Can incorporating CoT reasoning enhance interpretability and credibility?
3. Can multi-stage reasoning structures reduce performance fluctuations across different CDR grading tasks, thereby enhancing prediction consistency?
The diagnosis of Alzheimer’s disease (AD) has traditionally relied upon neuropsychological assessments, imaging examinations (such as MRI and PET), and cerebrospinal fluid biomarkers [
The advent of LLMs, such as ClinicalBERT, LLaMA, and domain-adapted models, has significantly advanced medical natural language processing. Recent reviews on LLMs in healthcare highlight their broad applicability across tasks including entity extraction, report summarisation, and outcome prediction [
Recent clinical text studies on LLMs have primarily focused on information extraction and classification tasks, typically providing only classification results or evidence snippets without systematic generative clinical reasoning chains. Simultaneously, many validation studies suffer from small data scales (e.g., evaluating a limited number of manually reviewed cases) or are constrained by single-center, controlled settings, making it difficult to cover complex real-world clinical contexts [
CoT prompts represent a recently developed methodology within large language model research, requiring models to articulate intermediate reasoning steps rather than directly outputting answers [
Strengthening CoT by sampling diverse reasoning paths and aggregating consensus provides a principled approach to reducing variability in clinical reasoning chains [
This study aims to construct an interpretable automated CDR grading system based on a two-stage experimental framework. The first stage focuses on establishing multiple high-performance classification baselines to provide impartial performance benchmarks. The second stage introduces the core innovation that a CoT Alzheimer’s Disease Diagnostic system to address the transparency issues of traditional black-box models in complex clinical reasoning.
| 0.5 | “insidious onset with progressive poor memory forgets conversation details but able to manage home affairs; occasional confusion reported by spouse.” | “Autistic thinking (+) vague responses; mild impairment in orientation and memory domains; functional independence maintained.” |
| 1.0 | “WITH daughter deterioration multiple complaints of forgetfulness, misplacing items, and poor concentration; sometimes fails to find way home.” | “Suspect depression (treated at Psyche) reports, mild to moderate decline, impaired attention span, partial insight preserved.” |
| 2.0 | “progressive for years with poor memory Forgetfulness noted by family members and reduced self-care; occasional urinary incontinence reported.” | “(favor) in progression with incontinence? cognitive decline with temporal disorientation, impaired judgment, and dependency for daily activities.” |
| 3.0 | “request application for disability certificate due to long-term confusion, unable to recognize relatives; total dependence for self-care.” | “Right PCA territory infarct (onset: )(TOAST type: ) consistent with severe dementia picture; bed bound, nonverbal, requires full assistance.” |
The raw data used in this study originated from a five year clinical dataset containing patients’ longitudinal EHR, primarily comprising the patient’s Subject (S) and clinical diagnostic Assessment (A) fields. To ensure data quality and analytical validity, we first executed a rigorous data cleaning process. Since multiple records might exist for the same patient across different time points, we utilized unique medical record identifiers and performed deduplication based on the longest text length principle. The initial dataset contained 745 raw patient records. Following preprocessing, all samples with empty assessment (A) fields were excluded, yielding 698 distinct and complete patient records suitable for subsequent modeling. The CDR labels in our dataset cover four clinically recognized Alzheimer’s disease severity levels: 0.5, 1.0, 2.0, and 3.0, corresponding to very mild, mild, moderate, and severe dementia, respectively. Each stage exhibits distinct functional characteristics: CDR 0.5 patients typically show mild memory deficits while maintaining daily independence; CDR 1.0 patients exhibit more pronounced cognitive and functional impairments; those with 2.0 require assistance for daily tasks; and those with 3.0 exhibit severe disorientation and complete dependence on caregivers.
This experimental design employs a systematic one-versus-one binary classification strategy, decomposing the complex multi-class CDR problem into a series of more specific and controllable diagnostic subtasks. The CDR scale is a widely recognized clinical standard for quantifying the degree of cognitive decline across multifunctional domains such as memory, orientation, judgment, and self-care.
To ensure consistency in assessment across different levels of disease severity, we constructed four binary classification subsets from the final dataset: 0.5 vs 1.0, 0.5 vs 2.0, 0.5 vs 3.0, and 1.0 vs 3.0. The 0.5 vs 1.0 subset contained 429 patient records, the 0.5 vs 2.0 subset contained 340 patient records, the 0.5 vs 3.0 subset contained 263 patient records, and the 1.0 vs 3.0 subset contained 358 patient records. Each subset was independently processed through the proposed CoT inference framework to evaluate diagnostic performance across varying degrees of cognitive impairment. Representative samples of these records are presented in Table . Each binary experiment focused on adjacent or clinically significant stage differences, enabling the system to precisely identify diagnostic boundaries. The Clinical Dementia Rating (CDR) itself reflects the progressive deterioration of multidimensional functions in Alzheimer’s disease patients, including memory, orientation, judgment, and self-care abilities. Therefore, the 0.5 vs 1.0 and 0.5 vs 2.0 comparisons assess whether the model can detect subtle early-stage differences in disease progression, while the 0.5 vs 3.0 and 1.0 vs 3.0 comparisons validate the model’s robustness across scenarios with significant functional gaps. Compared to directly constructing a multi-class classification model, decomposing the task into binary sub-tasks corresponding to clinically meaningful decision boundaries reduces label ambiguity and enhances interpretability. This grouping approach thus possesses clear medical significance while providing the model with well-defined, verifiable classification objectives. This enables systematic evaluation of reasoning stability and discriminative capability across different levels of cognitive decline.
For each paired task, the system dynamically extracts patient records corresponding to the two target CDR grades from the preprocessed dataset. These subsets are then divided into training (80%) and test (20%) sets, ensuring balanced representation across severity levels. This design not only simplifies the model’s decision space but also enables targeted evaluation of its reasoning stability and generalization capabilities under diverse clinical contrast conditions. Through this approach, the study systematically explores how the proposed reasoning framework adapts to varying degrees of cognitive impairment, revealing its diagnostic interpretability and sensitivity to disease progression.
We developed a four-stage CoT integrated diagnostic framework designed to simulate the collective reasoning process of expert clinical panels. This architecture overcomes the limitations of traditional single-threaded CoT prompts by integrating mechanisms for reasoning diversity, information fusion, logical calibration, and auditability. The entire system aims to enhance the interpretability and reliability of CDR grading, with the overall workflow illustrated in Figure and List .
The foundation of this framework is the CoT Generation and Diversity stage, serving as the core reasoning layer. For each patient record composed of subjective clinical notes (S) and a predefined binary CDR label pair (e.g., [0.5, 1.0]), the system initiates four independent reasoning processes. Each process is executed by a fine-tuned large language model. The model generates structured JSON-formatted reasoning outputs covering six diagnostic domains—memory, orientation, judgment and problem-solving, community affairs, family and hobbies, personal care—along with independent preliminary CDR scores. To ensure reasoning diversity and mitigate potential biases from deterministic reasoning, each reasoning attempt employs an independent random seed. This deliberate randomness, termed seed cracking, fosters diverse interpretive perspectives while maintaining consistency in the medical reasoning context. The system instantly rejects any JSON outputs with formatting errors or invalid structures, ensuring only complete analyses proceed to the next stage. This phase yields four fully independent, semantically rich diagnostic reports, each representing a unique reasoning pathway.
These four assessment results are treated equally. All evaluation texts are subsequently fed into a language model, which synthesizes the texts into a single coherent diagnostic narrative from a physician’s perspective. This process integrates recurring clinical clues while resolving contradictions. The final CDR grade is determined by classifying this consolidated narrative. This means the final decision originates from the interpretation of the integrated text, rather than a direct aggregation of the initial scores.
The Final Classification and Logical Consistency Check stage transforms the integrated assessment into an authoritative diagnostic judgment. Here, the model assumes the role of a CDR scoring expert, generating final scores based on predefined label sets. To ensure robustness and data integrity, the system programmatically extracts predicted scores from model text responses using strict regular expression patterns. Should the model generate out-of-range values (e.g., 0.0 or 2.0), the system automatically executes a clamping operation, adjusting the score to the nearest valid label (e.g., [0.5, 1.0]). This mechanism mitigates rare yet potentially disruptive prediction errors while preserving the integrity of valid reasoning chains.
In the final stage, summary Generation and Auditability Enhances interpretability by translating complex multi-stage reasoning into transparent, reviewable narratives. The system re-engages the model as a senior clinical consultant, synthesizing four initial CoT reports, evaluations, and final classification outcomes to summarize the entire diagnostic process. The generated textual audit trail provides clinicians with clear visibility into model reasoning transparency, ensuring each diagnostic conclusion is traceable to its underlying evidence, with the overall output example illustrated in Figure .
In summary, this four-phase CoT integration framework transforms traditional black-box LLM reasoning into a transparent, verifiable diagnostic process. By integrating reasoning diversity, structured aggregation, logical calibration, and explicit reasoning, the system achieves high robustness and interpretability, establishing itself as a reliable and explainable clinical AI solution for Alzheimer’s disease assessment.
Under the one-versus-one binary classification strategy, we conducted a comprehensive evaluation of model performance to reveal the actual capabilities of different models in distinguishing subtle CDR levels. Table compares zero-shot and fine-tuned models based on large language models for CoT diagnostic system.
| 0.5 vs. 1 | QWEN2-7B zero shot prompting | 0.55 | 0.49 | 0.39 | 0.58 | 0.51 |
| Microsoft Phi-3B (CoT) | 0.23 | 0.50 | 0.32 | 0.46 | 0.50 | |
| QWEN3-4B (CoT) | 0.43 | 0.45 | 0.42 | 0.49 | 0.45 | |
| QWEN2-7B (CoT) | ||||||
| 0.5 vs. 2 | QWEN2-7B zero shot prompting | 0.64 | 0.53 | 0.42 | 0.54 | 0.53 |
| Microsoft Phi-3B (CoT) | 0.59 | |||||
| QWEN3-4B (CoT) | 0.58 | 0.53 | 0.45 | 0.56 | 0.53 | |
| QWEN2-7B (CoT) | 0.29 | 0.45 | 0.35 | 0.54 | 0.45 | |
| 0.5 vs. 3 | QWEN2-7B zero shot prompting | 0.62 | 0.54 | 0.41 | 0.47 | 0.54 |
| Microsoft Phi-3B (CoT) | 1.00 | 0.33 | 0.50 | 0.33 | NaN | |
| QWEN3-4B (CoT) | 0.40 | 0.47 | ||||
| QWEN2-7B (CoT) | 0.54 | 0.54 | 0.54 | |||
| 1 vs. 3 | QWEN2-7B zero shot prompting | 0.50 | 0.50 | 0.44 | 0.44 | 0.50 |
| Microsoft Phi-3B (CoT) | 0.56 | 0.56 | 0.56 | |||
| QWEN3-4B (CoT) | 0.46 | 0.47 | 0.32 | 0.32 | 0.47 | |
| QWEN2-7B (CoT) | 0.50 | 0.50 |
To evaluate the effectiveness of the proposed CoT framework, we conducted a comprehensive comparison across multiple LLMs under two distinct settings: standard zero-shot prompting, involving single-shot inference without explicit reasoning steps, and CoT-enhanced reasoning, involving stepwise intermediate reasoning. Experimental results are presented in Table and F1 scores across different models are shown in Figure . Findings demonstrate significant performance gains through CoT integration, particularly in distinguishing binary classification tasks across different CDR levels.
The CoT-based diagnostic system outperformed traditional prompting methods across most evaluation metrics, achieving superior F1 scores, accuracy, and AUC values. For instance, in the 0.5 vs. 1.0 classification task, the Qwen2-7B (CoT) model achieved an F1 score of 0.54 and accuracy of 0.61, outperforming the zero-shot version (F1 = 0.39, accuracy = 0.58) by +0.15 F1 score, demonstrating the effectiveness of explicit reasoning in fine-grained clinical differentiation. Similarly, the Qwen3-4B (CoT) model showed substantial improvement in fine-grained discrimination tasks like 0.5 vs. 2 (F1 = 0.45, accuracy = 0.56). These results indicate that structured reasoning effectively enhances models’ ability to distinguish borderline dementia stages. The Qwen2-7B (CoT) model exhibits the most stable performance characteristics, maintaining balanced precision and recall across all task pairs. Notably, both metrics exceed 0.57 in the 1 vs 3 classification task. This stability indicates the model’s ability to handle broader CDR variations while preserving decision reliability. In contrast, the Qwen3-4B (CoT) model performs stronger on moderate classification tasks (e.g., 0.5 vs 2) but shows declining performance on more distant class pairs (e.g., 1 vs 3), suggesting that its smaller parameter size may limit its generalization ability across varying degrees of severity.
In contrast, the Phi-3B (CoT) model exhibits significant performance fluctuations, revealing its limitations in complex diagnostic reasoning tasks. Although the model achieves competitive recall rates in certain subtasks, its precision and AUC values fluctuate dramatically, indicating unstable decision boundaries and inconsistent reasoning chains. For instance, the Phi-3B model achieved an F1 score of 0.56 in the 0.5 vs 2 task, yet its performance plummeted to 0.50 in the 0.5 vs 3 task. These inconsistencies suggest that small-scale models may struggle to maintain logical coherence across multi-step reasoning sequences, highlighting the need for sufficient model capacity to ensure the stability of CoT-based reasoning in medical diagnostic scenarios.
Analysis of cross-model F1 score trends reveals model-scale effects and algorithmic influences. Qwen2-7B (CoT) consistently maintains the highest and most stable F1 score performance across all CDR tasks, indicating that greater model capacity enables construction of more coherent reasoning chains and enhances diagnostic discrimination capabilities. In contrast, Qwen3-4B (CoT) shows moderate gains at moderate discrepancy levels (0.5 vs 2) but exhibits weaker stability on distant category pairs (1 vs 3), indicating limited generalization in scenarios with severe cognitive discrepancies. Meanwhile, Phi-3B (CoT) exhibits fluctuating F1 scores, reflecting the sensitivity of smaller architectures to reasoning depth and task complexity. These trends collectively demonstrate that parameter scale and CoT prompt design synergistically enhance reasoning fidelity and stability. Notably, models exceeding 7 billion parameters appear to strike a critical balance between reasoning diversity and consistency, yielding more reliable diagnostic outputs.
Simultaneously, we observed the performance of multi-stage CoT frameworks on heterogeneous CDR classification tasks. Although absolute F1 scores fluctuated with task difficulty, CoT-enhanced models (particularly Qwen2-7B) avoided failure and maintained balanced precision-recall curves across all four binary settings. In contrast, the smaller Phi-3B (CoT) model exhibited significant instability, indicating poor discrimination capabilities under the most challenging scenarios. These results demonstrate that multi-stage reasoning structures, when combined with sufficient model capacity, can mitigate extreme performance fluctuations and maintain consistent decision boundaries across both adjacent and distant CDR comparisons.
CoT reasoning enhances diagnostic consistency and interpretability. Unlike traditional LLM prompts relying solely on implicit statistical correlations, CoT-based reasoning generates transparent intermediate reasoning paths that mirror clinicians’ cognitive processes during dementia assessments. Furthermore, the multi-stage in CoT reasoning mitigates output instability by enforcing internal logical consistency between reasoning steps. In contrast, zero-shot prompts lacking CoT guidance often yield inconsistent and less reliable predictions due to the absence of structured reasoning supervision.
Experimental results validate the effectiveness of integrating CoT reasoning into large language models for Alzheimer’s disease assessment. Compared to existing diagnostic frameworks based on CoT or reasoning, this system focuses on structured electronic health record analysis rather than speech or synthetic datasets [
This study demonstrates that integrating CoT reasoning into large language models significantly enhances their diagnostic stability and interpretability in CDR assessments. Beyond numerical performance gains, the CoT framework transforms conventional large language models into structured reasoning systems capable of articulating clinical evidence, weighing diagnostic clues, and integrating multi-perspective judgments. Consistent improvements observed across both Qwen2-7B and Qwen3-4B models indicate that CoT integration offers benefits independent of specific model architectures.
The CoT-based diagnostic system introduces a transparent reasoning mechanism aligned with clinical expert decision-making processes. By explicitly generating intermediate reasoning paths and employing multi-stage evaluation, this system transforms the previously opaque prediction process into a traceable, verifiable chain of reasoning, significantly enhancing the model’s credibility in medical applications. Simultaneously, the multi-agent design effectively mitigates the randomness inherent in large language models during reasoning, leading to more stable performance on complex borderline tasks.
It should be noted that while CoT reasoning incurs additional computational overhead (due to multiple inference steps), this trade-off is acceptable as the resulting gains in diagnostic reliability and interpretability hold significant value for clinical applications. This approach bridges automated prediction with explainable AI, providing a scalable technical foundation for future deployment in real-world medical settings.
Although these findings are encouraging, the study has limitations. The dataset size is relatively small, and the analysis relies solely on text-based electronic health records (EHRs), excluding imaging data such as magnetic resonance imaging (MRI) or positron emission tomography (PET). Furthermore, external validation using independent cohorts is still required to confirm its generalizability. Future research will explore multimodal integration (e.g., combining EHR and MRI features) and human-machine collaborative validation frameworks to enhance interpretability and clinical reliability.
This study proposes a CoT reasoning framework based on large language models for the clinical assessment and diagnosis of AD. Unlike traditional black-box fine-tuning approaches, this system explicitly simulates the clinical physician’s reasoning process by generating intermediate diagnostic inferences and multi-layer evaluations. Experimental results demonstrate that the proposed CoT diagnostic system not only significantly enhances model interpretability and decision transparency but also exhibits higher consistency and stability in distinguishing between adjacent CDR grades. Enhancing reasoning transparency not only improves interpretability at the individual diagnostic level but also establishes a scalable framework for trustworthy clinical artificial intelligence. This advancement facilitates the integration of interpretable large language models into healthcare systems.
By transforming the model’s reasoning process into a structured, traceable chain of reasoning, this study establishes a novel connection between automated prediction and explainable artificial intelligence, providing a scalable technical foundation for large language models in clinical decision support systems.
Future research will further expand the CoT reasoning framework and integrate longitudinal EHR information to achieve multimodal diagnostic fusion. This will enable a more comprehensive characterization of disease dynamics and advance precision medicine applications in dementia diagnosis. Additionally, external validation using multicenter datasets will be conducted, alongside exploring the design of human-machine collaborative reasoning systems, laying the groundwork for practical deployment in clinical settings.