The authors propose a novel layer to perform non-linear transformations inside a neural network. They design a Vector Field Layer which learns a vector field to be applied to the input.

Let $$X$$ be the input, $$\mu_{i}$$ the means of the $$S$$ gaussians and $$V_{i}$$ the $$S$$ learned vectors.

We can compute $$X_{i+1} = X_{i} + h(K(X_{i}, \theta))$$, the transformed input.

$$h$$ is the step-size, $$\theta$$ are the parameters $$(\mu_{i}, V_{i})$$

$$K(X_{i}, \theta)$$ is a weighted sum of vectors, the weights are a result of a kernel function. (see eq. 2-3).

While this is early work, it’s a really interesting method.