Vaussenat, Fabrice, Bhattacharya, Abhiroop, Payette, Julie, Desmettre, Thibaut, Saidi, Alireza, Gagnon, Ghyslain et Cloutier, Sylvain G..
2026.
« NeuroPlast: a learnable activation function evaluated under knowledge distillation for medical image classification ».
Frontiers in Artificial Intelligence, vol. 9.
Prévisualisation |
PDF
Gagnon-G-2026-34512.pdf - Version publiée Licence d'utilisation : Creative Commons CC BY. Télécharger (2MB) | Prévisualisation |
Résumé
Deep neural networks for medical image classification rely almost exclusively on fixed activation functions such as ReLU. We introduce NeuroPlast, a parametric activation function whose four differentiable components (a shifted sigmoid modeling NMDA-type voltage gating, a Gaussian plateau inspired by AMPA receptor scaling, an excitatory rectifier, and an inhibitory leak) are combined through six learnable parameters. Two mixing strategies are evaluated: a static variant with fixed learned weights, and a metaplastic variant whose mixing coefficients adapt per sample via a lightweight squeeze-excite gate conditioned on channel statistics. NeuroPlast is embedded within NADN Ultra, a 13.5 M-parameter residual convolutional neural network (CNN) with convolutional block attention modules (CBAM), trained entirely from scratch through a two-phase knowledge distillation (KD) pipeline. The teacher is a fine-tuned EfficientNet-B0; the student combines logit-level KD with optional feature-level alignment losses. Across four medical imaging benchmarks (Brain Tumor MRI, 7,200 images, 4 classes; Chest X-ray Pneumonia, 5,856 images, 2 classes; Skin Cancer HAM10000, 10,015 images, 7 classes; COVID-19 Radiography, 10,848 images, 2 classes), evaluated under five-fold stratified cross-validation with 95% confidence intervals, the static-KD variant reaches 99.4–99.5% accuracy on COVID-19 X-ray across three seeds, on par with pretrained EfficientNet-B0 (99.18%, within replication noise) and above ResNet-18 (98.69%). It closes 65% of the accuracy gap on Brain Tumor MRI (98.29% vs. 99.13%), reaches 92.5% on Chest X-ray under a uniform class-balancing rule, and 79.5% on Skin Cancer under lesion-grouped cross-validation. The metaplastic variant achieves 99.28% on COVID-19 X-ray, still above pretrained baselines, but does not consistently outperform the static version, a negative result analyzed through ablation experiments. On three tabular medical datasets spanning three orders of magnitude in sample size (569 to 253,680), NeuroPlast matches five established activations within ±1.7 percentage points; at 253 K samples all activations converge within 0.09 pp. Our findings indicate that knowledge distillation is the primary enabler for from-scratch architectures to approach pretrained-level performance on medical images; under an identical distillation pipeline, NeuroPlast adds a small but consistent gain over ReLU, GELU, Swish, Mish, and PReLU, leading on all four imaging benchmarks by margins below one point.
| Type de document: | Article publié dans une revue, révisé par les pairs |
|---|---|
| Chercheur(-euse): | Chercheur(-euse) Gagnon, Ghyslain Cloutier, Sylvain G. |
| Affiliation: | Génie électrique, Génie électrique |
| Date de dépôt: | 02 oct. 2026 20:51 |
| Dernière modification: | 02 oct. 2026 23:37 |
| URI: | https://espace2.etsmtl.ca/id/eprint/34512 |
Actions (Authentification requise)
 |
Dernière vérification avant le dépôt |