Why your spectral classifier is probably overfitting, and how to prove it isn't

A familiar pattern: a Raman or SERS classifier reports 98 percent accuracy in development and then falls apart on the next batch of samples. The instinct is to reach for a more powerful model. In my experience the problem almost always lies upstream, in how the data was split and how the pipeline was validated. Spectroscopy is unusually prone to a specific failure, where the model learns the identity of the sample or the batch rather than the chemistry, and standard validation hides it.

The structure of spectral data invites leakage

Spectra are cheap to acquire in replicate. One physical sample, one patient, or one production batch typically yields many highly correlated spectra. If you shuffle all spectra together and take a random train and test split, replicates of the same sample land on both sides of the split. The model can then reach high test accuracy by recognising which sample a spectrum came from, a quantity that will never be available at prediction time. The reported number is real, and it is meaningless.

The fix is to split at the level of the independent unit, not the spectrum. If the unit of decision is a patient, a batch, or a material lot, then every spectrum from that unit must sit entirely in training or entirely in test. Group-aware cross-validation, using grouped folds, leave-one-batch-out, or leave-one-patient-out, enforces this. The rule is simple: the test set must contain units the model has never seen in any form.

Preprocessing is part of the model

A quieter leak comes from preprocessing. Baseline correction parameters, normalisation scalers, global mean-centering, SNV or EMSC statistics computed across the whole dataset, PCA, and feature selection all learn something from the data. Fit any of them on the full dataset before splitting, and test information flows into training. The accuracy inflates, and it inflates in a way that survives a group-aware split, so it is easy to miss.

Everything that learns from data belongs inside the resampling loop, fitted on the training fold only and applied to the held-out fold. In practice this means wrapping the whole chain, from baseline through classifier, in a single pipeline and cross-validating the pipeline rather than the model alone. Band or feature selection is the most common offender: choosing discriminative wavenumbers using all samples and then validating on a subset is circular, and it produces confident nonsense.

The confounder you built into the experiment

The most damaging leaks are designed into the acquisition, not the analysis. If cases were measured on Monday and controls on Tuesday, or on different instruments, or with different substrate lots, the model can separate the classes perfectly by learning the acquisition condition. It has found a real signal, just not the one you wanted, and it will not transfer.

Randomise acquisition order across classes, interleave batches, and record instrument, day, operator, and substrate lot as metadata. Then test the model on data acquired on a different day, a different instrument, or a different substrate batch. The gap between within-batch and across-batch performance is one of the most informative numbers you can report, and it is the one most often omitted.

A validation hierarchy that holds up

Three layers, in order of increasing honesty. Group-aware cross-validation for internal estimates. Nested cross-validation when you tune hyperparameters, so the reported error is not contaminated by model selection. And an external test set acquired independently, ideally at a different time and on a different instrument, as the final word. For a method destined for quality control or the clinic, the external, independently measured set is not optional. It is the only estimate that predicts field performance.

Two cheap diagnostics catch most trouble. A label-permutation test: retrain on shuffled labels, and if accuracy stays well above chance, you have leakage or overfitting. And the within-versus-across-batch comparison already mentioned: a large drop across batches means the model is riding a confounder.

Report the right n

Statistical power is governed by the number of independent units, not the number of spectra. A study with 2,000 spectra from 12 patients has an effective sample size closer to 12. Report both numbers, and size any claim of accuracy against the smaller one.

Checklist

• Split at the level of the independent unit, never the spectrum.
• Fit all preprocessing inside the resampling loop, on training folds only.
• Randomise acquisition across classes; record batch, instrument, day, operator, and lot.
• Use nested cross-validation when tuning hyperparameters.
• Validate on an independently acquired batch or instrument.
• Run a label-permutation test and report within-versus-across-batch performance.
• Report the number of independent samples, not just the number of spectra.