Structured Confidence Perturbation via Fisher Geometry for Adversarial Training
THE QUICK READ
The paper argues that adversarial training suffers from pseudo-attacks, perturbations that raise loss without truly altering the model's confidence structure. The authors propose two complementary approaches: a hybrid objective combining cross-entropy and KL divergence, and a geometric reconstruction of adversarial training that maximizes distribution distance on the confidence manifold. They claim the optimal perturbation direction aligns with principal eigenvectors of the Fisher information matrix, plus an efficient algorithm avoiding high-dimensional matrix computation.
KEY POINTS
- Motivation is pseudo-attacks in adversarial training: loss rises but confidence structure is not truly changed
- A hybrid objective combines cross-entropy and KL divergence to disrupt both prediction accuracy and distribution consistency
- Theory claims the optimal perturbation direction aligns with principal eigenvectors of the Fisher information matrix, with an algorithm avoiding high-dimensional matrix computation
READING LIMITS
Based only on the title and abstract; method details, experimental setup and numbers were not examined, so the theoretical analysis and reported results cannot be verified, and all superiority claims are the authors' own.
Source abstract [EN]
Adversarial training is often plagued by "pseudo-attacks", perturbations that significantly increase the loss value but fail to truly alter the model's confidence structure. To address this, we propose two complementary approaches to directly attack and defend against confidence structure deformation. First, building upon the traditional loss maximization perspective, we introduce a hybrid objective that integrates cross-entropy and KL divergence, encouraging adversarial examples to simultaneously disrupt prediction accuracy and distribution consistency. Second, we geometrically reconstruct adversarial training by generating perturbations that maximize the distribution distance on the confidence manifold. Theoretical analysis shows that the optimal perturbation direction aligns with the principal eigenvectors of the Fisher information matrix. We also propose an efficient algorithm to avoid high-dimensional matrix computations. Experiments on multiple datasets and model architectures demonstrate that our method outperforms existing methods in adversarial robustness and robust transferability, particularly excelling in structural perturbations within high-confidence regions.
AI summaries use public titles and abstracts only. Research claims belong to the authors; consult the original paper.