Publications

DualTower-FT with an Adaptive Runtime Safeguard: A Deep Tabular Approach for ICU Stroke Mortality

Zhengrong Jia*, Kwong-Cheong Wong* Accepted · Short Paper

The 23rd Pacific Rim International Conference on Artificial Intelligence (PRICAI 2026), Guangzhou, China, November 16–20, 2026.

Code

Status

Accepted as a short paper at PRICAI 2026, the 23rd Pacific Rim International Conference on Artificial Intelligence (CORE Rank B), to be held in Guangzhou, China, November 16–20, 2026. The proceedings will be published by Springer in the Lecture Notes in Artificial Intelligence (LNAI) series. The code is linked above; the paper will be linked here once it is available.

Overview

DualTower-FT extends the Dual-Tower Transformer of our CCAI 2026 paper for in-hospital mortality prediction in ICU stroke patients. It changes two things. The numerical tower now aggregates its tokens through an FT-Transformer-style [CLS] token instead of mean pooling, and the Adaptive Runtime Safeguard moves into the model's forward pass, so that NaN or infinite values, out-of-range categorical indices and numerical extremes cannot crash inference. On 11,187 stroke patients from 176 eICU hospitals, DualTower-FT reaches an AUPRC of 0.6247 ± 0.0015 over five seeds on the held-out test cohort, the highest among the deep learning models compared, while XGBoost keeps the overall lead (0.6467). The paper also reports a negative result: none of the MIMIC-IV transfer strategies tested improved on training with eICU alone.

Model Comparison

Model AUPRC
DualTower-FT (ours) 0.6247 ± 0.0015
XGBoost 0.6467
Random Forest 0.6236
Dual-Tower Transformer (CCAI 2026) 0.6171 ± 0.0058
Neural Network 0.5394 ± 0.0054
Standard Transformer 0.5279 ± 0.0195
Standard MLP 0.5170 ± 0.0081

AUPRC is the primary metric at the cohort's 14.0% mortality rate; DualTower-FT's AUROC is 0.8848, against 0.8908 for XGBoost. DualTower-FT: mean ± SD over 5 random seeds on the 94-feature cohort (test N = 2,238). The other rows are the CCAI 2026 results on an earlier 97-feature cohort and were not retrained, so the differences are observed rather than controlled. XGBoost and Random Forest are single deterministic runs.

Architecture & Methodology

  • Categorical Tower — Learnable embeddings for gender, ethnicity and hospital site, as in the CCAI 2026 model
  • Numerical Tower (FT-Transformer) — Each of the 91 numerical features becomes a 64-dimensional token; a learnable [CLS] token is prepended and the sequence passes through a two-layer Pre-LayerNorm Transformer encoder with four attention heads. The [CLS] output replaces mean pooling
  • Fusion — Tower outputs concatenated (22 + 64 dimensions) into an MLP head
  • Adaptive Runtime Safeguard — A preprocessing layer imputes missing values with training medians and rejects any NaN left over; a forward-pass layer then replaces NaN and ±∞ with 0, clamps each categorical column to its embedding vocabulary, and clamps numerical features to ±10 in z-score units

Safeguard Stress Test

Injected into 1% of test rows Safeguard on Safeguard off
Nothing (clean input) 0.6246 ± 0.0014 Crash
Age = 200 0.6242 ± 0.0015 Crash
Glucose at 10× its observed maximum 0.6249 ± 0.0012 Crash
NaN in SaO2 0.6234 ± 0.0013 Crash
BUN = −1 0.6253 ± 0.0016 Crash

AUPRC over 5 seeds. With the safeguard on, no batch crashed or returned a NaN prediction in any condition; with it off, every batch raised an exception. All four injections hit numerical columns. Clean input also fails without the safeguard because the categorical clamp is what maps the scaled categorical columns back to valid embedding indices, so it runs on every input. The guarantee is that a finite prediction is always produced, not that it is clinically correct.

MIMIC-IV Transfer

Training regime AUPRC
eICU only (reference) 0.6247 ± 0.0015
eICU + MIMIC-IV pooled 0.6188 ± 0.0080
Pooled pre-training, then eICU fine-tuning 0.6169 ± 0.0066
As above, with a source-domain embedding 0.6226 ± 0.0048
Source-domain architecture, eICU only (control) 0.6240 ± 0.0057

All rows are tested on the same eICU test cohort. None of the three transfer strategies improved on eICU-only training. The best, with a source-domain embedding, was 0.14 pp below its eICU-only control, less than one seed's SD. Two likely causes, not yet isolated: six eICU measurements are missing from MIMIC-IV, and MIMIC-IV is a single medical centre while eICU spans 208 hospitals.

Dataset & Reproducibility

eICU Collaborative Research Database v2.0 — 11,187 stroke patients from 176 of its 208 hospitals, split at patient level into 7,830 training, 1,119 validation and 2,238 test patients, stratified by in-hospital mortality (14.0% in every split). Each patient has 94 features: 3 categorical (gender, ethnicity, hospital site) and 91 numerical (age, vital signs and laboratory values). MIMIC-IV v3.1 adds 7,125 patients from a single academic medical centre for the transfer experiments. Both datasets require PhysioNet credentialed access.

Code — Dual-Tower-Transformer-eICU-Stroke-FT: DualTower-FT and its runtime safeguard, the stress-test harness, two-stage MIMIC-IV transfer, the ablation variants, a five-seed runner and the per-seed result files. It builds on the CCAI 2026 code release.

Back to Publications