Summarize Gaussian-process uncertainty calibration — gp_calibration

Summarizes exact LOO predictive diagnostics. The returned statistics are descriptive diagnostics rather than a pass/fail calibration test.

Usage

gp_calibration_diagnostics(model, interval_level = 0.95)

Arguments

model: A fitted gaussianprocesses_model.
interval_level: Probability level for LOO prediction intervals.

Value

A named list of calibration summaries.

Details

PIT values should be assessed as a distribution rather than individually. Under a correctly specified continuous predictive distribution they are uniform in repeated sampling.

Stability

Experimental: this interface may change in a minor release, and every change is listed in NEWS. See gaussianprocesses-package for the policy.

Examples

x <- seq(-2, 2, length.out = 20)
model <- fit_gp(
  x,
  sin(2 * x),
  kernel = rbf_kernel(length_scale = 0.6),
  noise_variance = 0.01
)

gp_calibration_diagnostics(model, interval_level = 0.9)
#> $interval_level
#> [1] 0.9
#> 
#> $empirical_coverage
#> [1] 1
#> 
#> $coverage_error
#> [1] 0.1
#> 
#> $mean_standardized_residual
#> [1] -6.955157e-18
#> 
#> $sd_standardized_residual
#> [1] 0.1568656
#> 
#> $mean_absolute_standardized_residual
#> [1] 0.08083751
#> 
#> $pit_mean
#> [1] 0.5
#> 
#> $pit_variance
#> [1] 0.003676772
#> 
#> $mean_log_predictive_density
#> [1] 1.037464
#>