Why "Adversarial Examples Are Features, Not Bugs" Should Change How We Define Model Robustness

Machine learning models don't fail because they're broken — they fail because they're doing exactly what they were trained to do, just not in the way we hoped. That tension sits at the heart of a growing debate in the robustness research community, and a response to the Ilyas et al. paper "Adversarial examples are not bugs, they are features" sharpens the argument considerably. You can follow the broader exchange in the main discussion article.

Adversarial examples as a symptom of a deeper problem

The Ilyas et al. hypothesis — that adversarial examples emerge from non-robust but genuinely useful features — doesn't stand alone. It fits neatly within a well-established body of work on distributional robustness, which has long argued that models fail under distribution shift because they latch onto superficial statistical patterns rather than meaningful structure. Adversarial examples, viewed through this lens, are simply the worst-case expression of that same tendency.

Supporting evidence comes from experiments that deliberately sidestep gradient perturbations altogether. One study trained and evaluated models using data processed through an extreme high-pass filter, stripping away everything that looks meaningful to a human eye. The resulting images appear entirely grayscale to people, yet models trained on these filtered inputs achieved 50% top-1 accuracy on ImageNet-1K. The features driving those predictions are real, naturally occurring, and essentially invisible without normalization — which raises an uncomfortable question about what our models are actually learning when they perform well on standard benchmarks.

Why adversarial training doesn't solve the robustness problem

Adversarial training is often treated as the go-to remedy for brittle models, but the picture is more complicated once you look beyond $\ell_p$ norm perturbations. When models are tested against a full Fourier basis of perturbations, a clear pattern emerges: naturally trained models hold up well against low-frequency noise but fall apart at mid-to-high frequencies. Adversarially trained models flip that profile — they gain robustness in the mid and high frequencies while becoming significantly more vulnerable to low-frequency corruptions.

The numbers make this concrete. Adversarial training drops performance on low-frequency fog corruption from 85.7% to 55.3%. Similar degradation appears with contrast and low-pass filtered noise. The implication is that adversarially trained models aren't actually more robust in any general sense — they've just shifted their reliance from one set of superficial statistics to another. That's a meaningful distinction, and one that current evaluation practices tend to obscure.

What this means for how the field measures and builds robustness

The practical consequence of all this is that the research community's fixation on small gradient perturbations has produced a narrow and somewhat misleading picture of model robustness. Models routinely degrade when evaluated on distributions that differ even slightly from their training data — under synthetic corruptions, natural distribution shifts, and conditions that have nothing to do with adversarial attacks. Current benchmarks largely don't surface these failure modes, which means the field may be optimizing for a metric that doesn't track real-world reliability.

The path forward, as argued here, requires broader and harder test sets that expose the full range of ways models can be fragile. Robustness to $\ell_p$-bounded perturbations is a small and largely detached subset of what security and real-world deployment actually demand. The Ilyas et al. team, in their response, broadly agrees — noting that expanding the perturbation set would help identify more non-robust features and push models toward the kinds of representations we actually want them to learn. Adversarial examples, in this framing, aren't a mystery to be solved in isolation. They're a signal that something more fundamental about how models generalize still needs to be addressed.

If you see mistakes or want to suggest changes, please create an issue on GitHub.

Diagrams and text are licensed under Creative Commons Attribution CC-BY 4.0 with the source available on GitHub, unless noted otherwise. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from …".

I can't discuss that. What I can help with is software development, code reviews, technical architecture, debugging, or anything else in my wheelhouse as a dev assistant. What are you working on?