Spectral-mixture kernels

A spectral-mixture kernel (Wilson and Adams, 2013) is a sum of $Q$ components with Gaussian spectral densities. With enough components it approximates any stationary covariance.

Usage

spectral_mixture_kernel(
  n_components = 1L,
  weights = 1/n_components,
  frequencies = 0,
  spectral_variances = 1
)

Arguments

n_components: Number of components $Q$.
weights: Positive component weights $w_q$: one value, recycled, or one per component.
frequencies: Frequencies $\mu_q$, in cycles per input unit: one value, one per component, or a matrix with one row per component and one column per input dimension.
spectral_variances: Positive spectral variances $v_q$, in squared cycles per input unit, shaped like frequencies.

Value

A kernel specification: one spectral component, or the sum of n_components of them.

Details

By Bochner's theorem a stationary kernel is the Fourier transform of a non-negative spectral density. The spectral mixture uses $S(s) = \sum_q \tfrac{w_q}{2} [N(s; \mu_q, V_q) + N(s; -\mu_q, V_q)]$, whose transform is $$k(\tau) = \sum_{q=1}^{Q} w_q \prod_{d=1}^{D} \exp(-2 \pi^2 \tau_d^2 v_{qd}) \cos(2 \pi \tau_d \mu_{qd}).$$ So $k(0) = \sum_q w_q$. A component with $\mu_q = 0$ is an RBF kernel with variance $w_q$ and length scale $\ell = 1 / (2 \pi \sqrt{v_q})$; a component with a small spectral variance is a long-lived oscillation with period $1 / \mu_q$.

Each component is a kernel with parameters weight, frequency, and spectral_variance, and the mixture is their sum_kernel(), so parameter paths are kernel1.weight, kernel1.frequency, and so on; a single component has no prefix. Frequencies are real-valued and optimized on the identity scale, the others on the log scale. Because $\cos$ is even, $-\mu$ and $\mu$ give the same kernel. optimize_gp() bounds every frequency by 0 and the Nyquist frequency $1 / (2 \Delta)$ of its input column, with $\Delta$ the smallest spacing between distinct inputs, and displaces it in later starts by one frequency-resolution bin, the reciprocal of the input range.

The likelihood is strongly multimodal: components can swap, merge, or settle on harmonics. Start from the data's spectrum with initialize_spectral_mixture(), and use several starts; the examples use n_starts = 5.

Stability

Experimental: this interface may change in a minor release, and every change is listed in NEWS. See gaussianprocesses-package for the policy.

References

Wilson, A. G. and Adams, R. P. (2013). Gaussian process kernels for pattern discovery and extrapolation. Proceedings of the 30th International Conference on Machine Learning, 1067–1075.

Examples

kernel <- spectral_mixture_kernel(
  n_components = 2,
  weights = c(1, 0.5),
  frequencies = c(0, 0.25),
  spectral_variances = c(0.01, 0.001)
)
kernel
#> SumKernel(
#>   SpectralComponent(weight=1, frequency=0, spectral_variance=0.01)
#>   SpectralComponent(weight=0.5, frequency=0.25, spectral_variance=0.001)
#> )

# k(0) is the sum of the weights.
evaluate_kernel(kernel, 0)
#>      [,1]
#> [1,]  1.5

# A component with frequency 0 is an RBF kernel.
all.equal(
  evaluate_kernel(spectral_mixture_kernel(1, 2, 0, 1 / (4 * pi^2)), 0:3),
  evaluate_kernel(rbf_kernel(variance = 2, length_scale = 1), 0:3)
)
#> [1] TRUE