Arham Doshi
Back
researchALSmachine-learningspeechsurvival-analysis

What an ALS trial database taught me about trusting a p-value

What an ALS trial database taught me about trusting a p-value

Where I'm working

I'm an undergrad research assistant in Dr. Jun Wang's lab at UT Austin (Speech, Language & Hearing Sciences and Neurology at Dell Medical School). The lab works on ALS speech biomarkers, silent speech interfaces, neural speech decoding from MEG, articulatory science, and deep-learning speech synthesis. (lab page)

I'm a CS + Linguistics double major, so speech data from ALS patients is close to the perfect overlap. My work so far splits into three tracks.

Track 1: a survival result that didn't survive

PRO-ACT is a pooled database of ALS clinical trials, about 13,000 patients. I started with a simple question: among people with limb-onset ALS, is there a survival gap between men and women?

Unadjusted, there is one (log-rank p = 0.022). Then I added covariates, and it dissolved. Any adjustment at all, and the gap was gone.

Unadjusted: log-rank p = 0.022. Adjusted for anything: gone.

Kaplan–Meier survival by sex, stratified by onset site. The gap only appears in the limb-onset panel (log-rank p = 0.0218); bulbar onset shows nothing (p = 0.97).Kaplan–Meier survival by sex, stratified by onset site. The gap only appears in the limb-onset panel (log-rank p = 0.0218); bulbar onset shows nothing (p = 0.97).

Two things explain most of it:

  • Baseline ALSFRS-R (a clinical rating of how much function a patient has) is the dominant predictor, at roughly a 9% lower hazard per point.
  • Limb-onset women in PRO-ACT are about 3.2 years older than the men at enrollment.

My read is that this is a trial enrollment artifact, not a biological sex effect. It's also not directly comparable to a 2024 Annals of Neurology paper that found a sex difference, because that one used a population-based registry rather than trial data (Grassano et al. 2024).

Track 2: can a voice tell you something?

The second track uses VOC-ALS, a public dataset of 102 ALS patients and 51 healthy controls with 40 pre-extracted acoustic features (Dubbioso et al. 2024).

I built a small pipeline and trained a random forest to separate ALS from controls. Pooled, it only reached an AUC of 0.64. The reason is that many ALS patients in the set don't have dysarthric speech yet, so their voices look like controls. Restricting to dysarthric patients, AUC rose to 0.758 (balanced accuracy 0.75), with pitch-variability features doing most of the work. That matches the dataset paper's main finding, which is a reassuring sanity check. Predicting severity was much weaker (Spearman ρ around 0.27 to 0.32).

Track 3: a prototype for blurring consonants

As ALS progresses, speakers blur contrasts like cat vs cap. Dr. Wang suggested trying AI to tell them apart, so I built a prototype on wav2vec2 plus a light classifier.

So far it's only been tested on healthy speakers from Google Speech Commands, with speaker-grouped 5-fold cross-validation: word macro-F1 of 0.914, place of articulation 0.900, voicing 0.961. Fine-tuning the whole network matched or beat the frozen version on 4 of 5 folds, but one fold collapsed and dragged the average down. Promising, but unstable at this sample size.

Word identity (4-way cat/dog/go/tree): macro-F1 0.914. Most confusions are between acoustically similar words.Word identity (4-way cat/dog/go/tree): macro-F1 0.914. Most confusions are between acoustically similar words.

Place of articulation (alveolar vs velar): macro-F1 0.900.Place of articulation (alveolar vs velar): macro-F1 0.900.

Voicing (voiced vs voiceless): macro-F1 0.961 — the easiest contrast of the three.Voicing (voiced vs voiceless): macro-F1 0.961 — the easiest contrast of the three.

What was hard

  • The clock starts at the wrong time. In PRO-ACT, survival is counted from trial enrollment, not symptom onset, so who gets enrolled shapes the result.
  • Messy data. The riluzole file is 99.6% zeros, so real treatment data has to come from the concomitant-medications file. Diagnostic delay only had about 9% coverage, so I couldn't use it as a mediator.
  • Heterogeneity. The non-dysarthric patients quietly diluted my classifier until I split them out.

What's next

  • Getting Dr. Wang's call on whether the survival work is a standalone paper or a stepping stone.
  • Moving the phonetic-contrast work to the lab's own ALS recordings. The public dataset I planned to use (TORGO) turned out to have only one ALS speaker.
  • Testing on true minimal pairs (cat/cad/cap/cab) so vowel differences don't sneak in.
  • Running an experiment where an LLM (Qwen3-8B) predicts 12-month mortality from PRO-ACT records, then comparing it to a standard random-forest baseline. It's designed but not run yet.

The takeaway

A p-value of 0.022 felt like a finding for about a day. What I learned is to ask what's different between the groups before asking whether the difference is real.