Highlights

The authors find two properties of neural networks.

There is no distinction between individual high-level units and random linear combinations of high-level units.
Existence and transferability of adversarial examples.

On the units

Let \(x \in \mathbb{R}^m\) be an input image and \(\phi(x)\) the activation values of some layer. One can look at what inputs maximize the features of \(\phi(x)\), that is:

The authors find that many images that satisfy

are semantically related to each other, where \(v\) is a random vector.

This puts into question the notion that neural networks disentangle variation factors across coordinates.

Adversarial examples

Let \(f:\mathbb{R}^m \to \{1,...,k \}\) be a classifier with an associated loss function. For a given input \(x \in \mathbb{R}^m\) and target label \(l \in \{1,...,k \}\), the aim is to solve

The minimizer is denoted \(D(x,l)\). This task is non-trivial only if \(f(x) \not= l\). The authors find an approximation of \(D(x,l)\) by line-search to find the minimum \(c>0\) for which the minimum \(r\) of the following problems atisfies \(f(x+r)=l\).

In the convex case this yields the exact solution.

Experiments

Existence of adversarial examples

Adversarial examples transfer to other architectures trained from scratch with different hyperparameters:

Adversarial examples transfer to other architectures trained on a disjoint training set:

Adding random noise to the input images is not as effective to misclassify samples when compared to adversarial examples generation:

Theoretical analysis

Conclusion

The semantic meaning of activations of neurons is not meaningful since random directions in feature space present similar properties.
Adversarial examples can be found for any neural network, they transfer across architectures trained with different hyperparameters and even different data sets.