Survival Analysis — Heart Failure Clinical Records

Author: Samakova Aiym (BDA 24O6)

Overview

This notebook analyzes the Heart Failure Clinical Records Dataset (Chicco & Jurman, 2020) — follow-up records of 299 heart failure patients admitted to the Faisalabad Institute of Cardiology, Pakistan, in 2015. It walks from first-principles survival analysis concepts up through classical statistical modeling and, in the extended sections, machine learning.

Dataset: heart_failure_clinical_records_dataset.csv (299 rows, 13 columns — 12 clinical features + DEATH_EVENT) Source: UCI Machine Learning Repository

Column	Description
age	Patient age (years)
anaemia	Decrease of red blood cells or hemoglobin (0/1)
creatinine_phosphokinase	Level of the CPK enzyme in the blood (mcg/L)
diabetes	Whether the patient has diabetes (0/1)
ejection_fraction	% of blood leaving the heart at each contraction
high_blood_pressure	Whether the patient has hypertension (0/1)
platelets	Platelets in the blood (kiloplatelets/mL)
serum_creatinine	Level of serum creatinine in the blood (mg/dL)
serum_sodium	Level of serum sodium in the blood (mEq/L)
sex	Woman (0) or man (1)
smoking	Whether the patient smokes (0/1)
time	Follow-up period (days)
DEATH_EVENT	Whether the patient died during follow-up (0/1)

Structure

Part	Content
1	Censoring concepts — death vs. censored counts, follow-up time distributions, why dropping censored patients biases results
2	Kaplan-Meier estimator built by hand on a 12-patient subset, with worked questions on risk sets and censoring
3	Overall KM survival curve; KM curves stratified by serum creatinine group; log-rank test
4	Additional stratified analyses (ejection fraction, diabetes) and discussion of the proportional hazards assumption
5 (bonus)	Multivariate Cox Proportional Hazards model — hazard ratios, forest plot, formal PH-assumption check (Schoenfeld residuals), and a stratified fix for the variable that violates it
6 (bonus)	Random Survival Forest (scikit-survival) — train/test comparison against Cox by concordance index, plus permutation feature importance

Key findings

• Independent risk factors (significant in both the Cox model and the Random Survival Forest): age, ejection fraction, serum creatinine. Anaemia and high blood pressure are also significant in the Cox model.
• Not significant once other variables are controlled for: diabetes, smoking, sex, platelets, creatinine phosphokinase, serum sodium.
• Proportional hazards assumption: violated by ejection_fraction (p ≈ 0.01); addressed by stratifying the Cox model on ejection-fraction bands, with negligible change to the other hazard ratios.
• Model comparison: Cox PH reaches a concordance index of ≈0.72 on held-out data; the Random Survival Forest reaches ≈0.74. The small gap suggests limited non-linear signal beyond what Cox already captures, favoring Cox for interpretability in a clinical context.

Requirements

pip install pandas matplotlib seaborn lifelines scikit-survival scikit-learn

Running the notebook

1. Open in Google Colab (originally authored there) or Jupyter.
2. Upload heart_failure_clinical_records_dataset.csv so it's available at /content/heart_failure_clinical_records_dataset.csv (Colab default), or edit the pd.read_csv(...) path in the first code cell to match your local location.
3. Run cells top to bottom. Part 5 requires lifelines; Part 6 requires scikit-survival (installed via a pip install cell within the notebook).

Files

• GROUP_2406_Aiym_extended.ipynb — the notebook (original Parts 1–4 plus bonus Parts 5–6)
• heart_failure_clinical_records_dataset.csv — dataset (not included here; download from the UCI link above or use your course-provided copy)