What We Will Cover

1. Motivation — why ICU mortality prediction is hard
2. Dataset & two core challenges
3. DT-Transformer: dual-tower architecture
4. Experiments — results & ablation
5. Theoretical implications (§V.B): why it works
6. Interpretability — what the model attends to
7. Conclusion & future directions

Paper CI5896 · DT-Transformer · eICU · Stroke Mortality Prediction

Why Standard Deep Learning Fails on Mixed Clinical Data

The challenge is not data volume — it is architectural mismatch between standard DL and the structure of clinical tabular data.

The eICU Database

Multi-center · freely available · Pollard et al., 2018

14% mortality → AUPRC is the right metric. Our 0.6171 is far above random-classifier level.

Two Problems We Had to Solve

Heterogeneous feature types demand separate encoders. Raw EHR data demands an online safety layer.

Our Model: The DT-Transformer

C1: Dual-Tower Encoder · C2: Adaptive Safeguard · C3: Optuna HPO

Dual-Tower encoding + Adaptive Safeguard + Optuna HPO — three contributions, one differentiable pipeline.

Left Tower: Categorical Embeddings

High-dimensional sparse indices → dense 22-d vector h_cat. Each feature gets its own learned embedding space.

Right Tower: Numerical Transformer

Self-Attention explicitly computes pairwise interactions across all 94 physiological features — something MLPs only approximate.

Feature Fusion and Prediction

The outputs of both towers are concatenated into h_joint ∈ ℝ⁸⁶ — bridging static demographic features and dynamic vital signs.

DT-Transformer Is Now Competitive With XGBoost

XGBoost retains a marginal AUPRC lead (0.6467) — our deep learning model closes the gap with full differentiability.

Self-Attention Is the Key: +14.41% Relative AUPRC Gain

Replacing the Transformer encoder with fully connected layers drops AUPRC by 14.41% — the gain is driven by self-attention modeling global feature interactions.

Why the Dual Tower Wins: Three Design Principles (§V.B)

Beyond empirical gains: three design principles for applying deep learning to heterogeneous clinical tabular data.

The Model Focuses on What Clinicians Care About

Top 5 features by attention weight

The model autonomously prioritizes clinically significant risk factors, improving interpretability compared to black-box baselines.

What We Showed and Where We Go Next

A fully differentiable alternative to gradient boosting — extensible to multimodal integration with unstructured clinical notes.

Thank You

Questions welcome

Zhengrong Jia · zhengrong.jia.academic@gmail.com

Paper CI5896 · CCAI 2026 · May 22–24, Nanjing