Adversarial Examples and Robust Evaluation: From FGSM to a scikit-learn Digits Experiment

Add a perturbation invisible to the human eye and a classifier calls a panda a gibbon — the example has been cited endlessly, and what it actually demonstrates often gets missed: that perturbation is not random noise, it is computed. An attacker follows the model’s own gradient, seeking precisely the direction in which loss rises fastest, which is why such a small change has such a large effect.

This article deconstructs the mathematical framework of gradient-based attacks (FGSM and PGD), provides PyTorch implementations for Red Teams, and details the production pipeline architectures required for adversarial defense.

1. The Mathematical Boundaries of Threat Models

An adversarial evaluation is mathematically meaningless without defining the feasible set of the attacker. The threat model is parameterized by:

• Attacker Knowledge: White-box (full access to \( \theta \), architectures, and gradients \( \nabla_x J \)) vs. Black-box (zero-th order optimization via queries).
• Perturbation Constraint (\( L_p \) Norm): The perturbation \( \delta \) is bounded by \( \|\delta\|_p \le \epsilon \). Common norms include \( L_\infty \) (maximum pixel change) and \( L_2 \) (Euclidean distance).
• Objective Function: Untargeted (\( \arg\max_\delta J(\theta, x+\delta, y) \)) vs. Targeted (\( \arg\min_\delta J(\theta, x+\delta, y_{target}) \)).

2. Fast Gradient Sign Method (FGSM)

FGSM is a single-step gradient-based attack that linearizes the loss function \( J \) around the input \( x \). Utilizing a first-order Taylor expansion, the attacker maximizes the loss under an \( L_\infty \) constraint.

The mathematical formulation is:

\[ \delta = \epsilon \cdot \text{sign}(\nabla_x J(\theta, x, y)) \]

\[ x_{adv} = \text{clip}(x + \delta, x_{min}, x_{max}) \]

PyTorch Implementation of FGSM

import torch
import torch.nn as nn

def fgsm_attack(model, images, labels, epsilon, criterion):
    images.requires_grad = True
    outputs = model(images)
    loss = criterion(outputs, labels)
    
    # Compute Jacobian / Gradients wrt input
    model.zero_grad()
    loss.backward()
    data_grad = images.grad.data
    
    # Create perturbation
    sign_data_grad = data_grad.sign()
    perturbed_images = images + epsilon * sign_data_grad
    
    # Project back to valid input domain (e.g., [0, 1])
    perturbed_images = torch.clamp(perturbed_images, 0, 1)
    return perturbed_images

3. Projected Gradient Descent (PGD)

While FGSM is computationally efficient, it underfits the adversarial objective. Projected Gradient Descent (PGD) is the universal first-order adversary. It solves the constrained optimization problem via iterative gradient steps, projecting the perturbation back onto the \( \epsilon \)-ball after each step.

The update rule for step \( t+1 \) is:

\[ x^{t+1} = \Pi_{x+\mathcal{S}} \left( x^t + \alpha \cdot \text{sign}(\nabla_x J(\theta, x^t, y)) \right) \]

Where \( \alpha \) is the step size and \( \Pi_{x+\mathcal{S}} \) is the projection operator onto the \( L_p \) ball.

PyTorch Implementation of PGD

def pgd_attack(model, images, labels, epsilon, alpha, iters, criterion):
    perturbed_images = images.clone().detach()
    # Random start within epsilon ball
    perturbed_images = perturbed_images + torch.empty_like(perturbed_images).uniform_(-epsilon, epsilon)
    perturbed_images = torch.clamp(perturbed_images, 0, 1)
    
    for _ in range(iters):
        perturbed_images.requires_grad = True
        outputs = model(perturbed_images)
        loss = criterion(outputs, labels)
        
        model.zero_grad()
        loss.backward()
        
        with torch.no_grad():
            adv_images = perturbed_images + alpha * perturbed_images.grad.sign()
            eta = torch.clamp(adv_images - images, min=-epsilon, max=epsilon)
            perturbed_images = torch.clamp(images + eta, 0, 1)
            
    return perturbed_images

4. Red/Blue Team Post-Mortem: Production Architecture Defenses

In production pipelines, basic “random noise” defenses are completely defeated by Expectation Over Transformation (EOT). Real-world mitigation relies on architectural integration:

• Adversarial Training Logic: The empirical risk minimization is modified to a min-max saddle point problem:
  
  \[ \min_\theta \mathbb{E}_{(x,y)\sim \mathcal{D}} \left[ \max_{\|\delta\|_p \le \epsilon} J(\theta, x+\delta, y) \right] \]
  Models are continuously trained on PGD-generated samples. This lowers the curvature of the loss surface but comes at the cost of the “accuracy-robustness trade-off” (diminished clean accuracy).
• Gradient Masking & Obfuscation (A Warning): Blue teams often inadvertently introduce shattered gradients (e.g., non-differentiable preprocessing). Red teams bypass this using Backward Pass Differentiable Approximation (BPDA). True defense requires verifying robustness via black-box transfer attacks.
• Inference Abstention & Out-of-Distribution (OOD) Detection: Deploying Mahalanobis distance metrics on deep feature representations to detect inputs lying far from the clean training manifold.

5. Robust Evaluation Reporting Standards

A production security audit must yield an evaluation matrix:

• Clean Accuracy vs. PGD-100 (100 iterations) Accuracy across a spectrum of \( \epsilon \) budgets.
• Evaluation of gradient-free attacks (e.g., SPSA) to certify that defenses are not merely relying on gradient obfuscation.
• System latency overhead introduced by dynamic OOD detection modules.

6. Robustness Audit Matrix

The strongest adversarial evaluation reports include both attack strength and defense side effects. A model should not be called robust unless the evaluation records the attack budget, adaptive checks, and production impact.

7. References

• Goodfellow, Shlens, Szegedy: Explaining and Harnessing Adversarial Examples
• Madry et al.: Towards Deep Learning Models Resistant to Adversarial Attacks (PGD)
• Athalye et al.: Obfuscated Gradients Give a False Sense of Security (BPDA)
• NIST AI 100-2e2025
• AI Security Lab README