Core Emulation in ExeUQ package
Daniel Williamson
28/11/2017
Building emulators: the precursor to calibration
This vignette aims to demonstrate the functionality of the emulators we have currently, highlighting the key steps and potentially highlighting things that need work.
Core steps and functionality (currently) with CanAM4
We begin by loading the key functions in the code and the CanAM4 data.
twd <- getwd()
source('BuildEmulator/BuildEmulator.R')## Loading required package: GenSA## Loading required package: far## Loading required package: nlme## far library : Modelization for Functional AutoRegressive processes## version 0.6-4 (2014-12-07)## Loading required package: fields## Loading required package: spam## Loading required package: dotCall64## Loading required package: grid## Spam version 2.1-1 (2017-07-02) is loaded.
## Type 'help( Spam)' or 'demo( spam)' for a short introduction
## and overview of this package.
## Help for individual functions is also obtained by adding the
## suffix '.spam' to the function name, e.g. 'help( chol.spam)'.##
## Attaching package: 'spam'## The following objects are masked from 'package:base':
##
## backsolve, forwardsolve## Loading required package: maps## Loading required package: lhs## Loading required package: mco## Loading required package: mvtnorm## Loading required package: ncdf4## Loading required package: parallel## Loading required package: rstan## Loading required package: ggplot2## Loading required package: StanHeaders## rstan (Version 2.16.2, packaged: 2017-07-03 09:24:58 UTC, GitRev: 2e1f913d3ca3)## For execution on a local, multicore CPU with excess RAM we recommend calling
## rstan_options(auto_write = TRUE)
## options(mc.cores = parallel::detectCores())## Loading required package: shape## Loading required package: tensorsource('HistoryMatching/HistoryMatching.R')
load("Demonstrations/globalCanAM4W1.RData")
head(tData)## t_IDs FACAUT UICEFAC CT CBMF WEIGHT
## 1 dma_1-000 -0.11111111 0.21739130 -0.09090909 -0.2000000 0.44444444
## 2 dma_1-001 0.93964656 0.21272724 0.44617789 -0.6328893 -0.12804819
## 3 dma_1-002 -0.49607751 0.49866598 0.66486451 -0.4184081 -0.57449420
## 4 dma_1-003 -0.01823092 0.87500017 -0.78226390 0.7481327 0.20387546
## 5 dma_1-004 0.18724746 -0.24641582 0.37947947 -0.9198253 -0.03209664
## 6 dma_1-005 -0.33415933 -0.04815128 -0.96533195 -0.2130182 -0.93268338
## DRNGAT RKXMIN XINGLE ALMC ALF TAUS1
## 1 -0.9000000 -0.63265306 -1.0000000 0.7781513 0.7993133 1.00000000
## 2 0.5548678 0.63519283 -0.2442266 0.9270296 0.5127372 0.20620020
## 3 -0.7147499 0.04806895 -0.6015328 0.1476252 -0.9153715 -0.45822211
## 4 0.7659221 -0.38371639 0.3084131 0.7993745 -0.7107337 -0.32409748
## 5 0.8924434 0.14729742 0.9987144 -0.7213836 -0.3770729 -0.02189904
## 6 -0.2679695 0.44082711 0.2129314 -0.5448322 -0.5037239 0.81073053
## CSIGMA CVSG BALX PCP OLR LPA CLDO
## 1 -0.33333333 0.1962080 0.916103 2.85956 240.029 0.291897 0.579167
## 2 0.90697659 -0.4331721 5.605330 2.95579 241.376 0.274161 0.586756
## 3 0.51120052 0.7759925 1.107120 3.14943 243.793 0.280279 0.525559
## 4 -0.03554562 0.1669205 6.872150 2.78977 244.308 0.261822 0.471185
## 5 -0.73212702 0.6142298 -4.677210 2.93444 236.977 0.317307 0.662686
## 6 0.08044063 -0.9508809 8.154230 2.68886 239.323 0.272699 0.512900
## CLDO20S MPCP1DTR MPCPC1DTR MPCPL1DTR VPCP1DTR VPCPC1DTR VPCPL1DTR
## 1 0.458043 4.56422 3.32959 1.234630 55.0539 17.2712 32.3932
## 2 0.505598 4.70510 3.16165 1.543450 54.4408 23.7380 22.9040
## 3 0.432856 5.04650 2.51922 2.527280 101.2060 12.9767 80.2562
## 4 0.346511 4.61732 3.53572 1.081590 41.5323 21.3537 14.4594
## 5 0.656435 4.69272 2.90937 1.783350 56.4616 15.4903 33.2680
## 6 0.406580 4.39895 4.01612 0.382828 50.9439 44.7553 3.5419
## ST STLND STUSA STBRA STL60S OBEG OBEGOCN WGT1
## 1 14.5623 8.55623 17.0945 28.1886 12.9034 1.40867 1.94342 29.7017
## 2 15.0599 9.82438 18.2459 28.4936 14.2373 6.05611 8.57419 28.3671
## 3 14.9696 9.18708 17.7589 26.7935 13.6190 1.67198 2.30910 29.9767
## 4 14.3284 7.97957 15.9793 24.7504 12.2772 7.26038 10.37870 28.8054
## 5 15.0301 9.65913 18.5349 27.8794 14.0057 -4.15018 -6.11184 28.5021
## 6 14.4430 8.23278 17.2718 25.8484 12.4941 8.57849 12.23910 29.1762
## WGT2 WGT3
## 1 79.6972 744.452
## 2 75.9896 711.998
## 3 79.3770 740.208
## 4 76.3718 712.879
## 5 75.9302 710.482
## 6 77.4365 722.800The first step is to set the data up so that a mean function can be built. Any mean function can be used, but we provide an advanced stepwise algorithm that has proved useful in many previous applications to climate models. We first generate a “Noise” variable and add it to the data. “Noise” is an insurance against over fitting. Our algorithm will try to add many different functions of combinations of inputs to our mean function. If anything involving “Noise” is added, we stop adding terms as we believe this indicates overfitting.
Noise <- rnorm(dim(tData)[1],0,0.5)
tData <- cbind(tData,Noise)We then tell the code which variables in the data frame (which we require to be named ‘tData’) are model inputs and we add “Noise” to this set.
cands <- names(tData)[2:14]
cands <- c(cands, "Noise")We then use the function EMULATE.lm to fit a mean function to pass to the emulator object. We should allow the user to specify their own mean function (via formula or lm-type object) or to use our algorithm.
myem.lm.canPCP <- EMULATE.lm(Response="PCP", tData=tData, tcands=cands, tcanfacs=NULL, TryFouriers=TRUE, maxOrder=2, maxdf = 20) ## [1] "Max Fourier reduction is 0.0277756600422503 using sin(pi*1*CT) and cos(pi*1*CT)"
## [1] "Max reduction is 0.0455585169939069 using CT"
## [1] "Max Fourier reduction is 0.0196674113205599 using sin(pi*1*UICEFAC) and cos(pi*1*UICEFAC)"
## [1] "Max reduction is 0.0459075880736152 using UICEFAC"
## [1] "Max Fourier reduction is 0.00668367883511844 using sin(pi*1*WEIGHT) and cos(pi*1*WEIGHT)"
## [1] "Max reduction is 0.0132736585961441 using WEIGHT"
## [1] "Max Fourier reduction is 0.00681530948074346 using sin(pi*1*CVSG) and cos(pi*1*CVSG)"
## [1] "Max reduction is with the Fourier term"
## [1] "Max Fourier reduction is 0.00202816222463734 using sin(pi*2*CT) and cos(pi*2*CT)"
## [1] "Max reduction is 0.00552932432741847 using CVSG"
## [1] "Max Fourier reduction is 0.00243919533316847 using sin(pi*1*UICEFAC) and cos(pi*1*UICEFAC)"
## [1] "Max reduction is 0.00351277696250736 using UICEFAC"
## [1] "Max Fourier reduction is 0.00193708619700064 using sin(pi*1*RKXMIN) and cos(pi*1*RKXMIN)"
## [1] "Max reduction is 0.00260091359172777 using RKXMIN"
## [1] "Max Fourier reduction is 0.00135838268141763 using sin(pi*1*FACAUT) and cos(pi*1*FACAUT)"
## [1] "Max reduction is 0.00267116583578417 using CBMF"
## [1] "Max Fourier reduction is 0.00107175357834954 using sin(pi*1*DRNGAT) and cos(pi*1*DRNGAT)"
## [1] "Max reduction is 0.00385691033176932 using ALMC"
## [1] "No further terms permitted with the given degrees of freedom"
##
## Call:
## lm(formula = PCP ~ CT + UICEFAC + I(UICEFAC^2) + WEIGHT + CVSG +
## RKXMIN + CBMF + ALMC + I(UICEFAC * CT) + I(WEIGHT * CT) +
## I(CVSG * CT) + I(RKXMIN * CT) + I(CBMF * CT) + I(ALMC * CT) +
## I(WEIGHT * UICEFAC) + I(CVSG * UICEFAC) + I(RKXMIN * UICEFAC) +
## I(CBMF * UICEFAC) + I(ALMC * UICEFAC) + I(CVSG * WEIGHT) +
## I(RKXMIN * WEIGHT) + I(CBMF * WEIGHT) + I(ALMC * WEIGHT) +
## I(RKXMIN * CVSG) + I(CBMF * CVSG) + I(ALMC * CVSG) + I(CBMF *
## RKXMIN) + I(ALMC * RKXMIN) + I(ALMC * CBMF) + I(sin(pi *
## CVSG * 1)) + I(cos(pi * CVSG * 1)), data = tData)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.031992 -0.008878 -0.001486 0.007815 0.029619
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.8716455 0.0045919 625.375 < 2e-16 ***
## CT 0.1882908 0.0051956 36.240 < 2e-16 ***
## UICEFAC 0.1411957 0.0049120 28.745 < 2e-16 ***
## I(UICEFAC^2) -0.0395684 0.0111086 -3.562 0.00134 **
## WEIGHT -0.0603384 0.0058736 -10.273 5.29e-11 ***
## CVSG 0.0497652 0.0091285 5.452 8.08e-06 ***
## RKXMIN -0.0241440 0.0048773 -4.950 3.18e-05 ***
## CBMF 0.0084529 0.0055697 1.518 0.14031
## ALMC -0.0093005 0.0049674 -1.872 0.07164 .
## I(UICEFAC * CT) 0.0332695 0.0122247 2.722 0.01105 *
## I(WEIGHT * CT) 0.0212076 0.0136957 1.548 0.13274
## I(CVSG * CT) -0.0126154 0.0109045 -1.157 0.25709
## I(RKXMIN * CT) 0.0005962 0.0092603 0.064 0.94913
## I(CBMF * CT) 0.0080121 0.0093460 0.857 0.39857
## I(ALMC * CT) -0.0149390 0.0102951 -1.451 0.15787
## I(WEIGHT * UICEFAC) -0.0053203 0.0090752 -0.586 0.56241
## I(CVSG * UICEFAC) -0.0218530 0.0116545 -1.875 0.07125 .
## I(RKXMIN * UICEFAC) -0.0095440 0.0091409 -1.044 0.30537
## I(CBMF * UICEFAC) 0.0248715 0.0092224 2.697 0.01172 *
## I(ALMC * UICEFAC) -0.0140469 0.0081873 -1.716 0.09727 .
## I(CVSG * WEIGHT) -0.0193827 0.0091355 -2.122 0.04285 *
## I(RKXMIN * WEIGHT) -0.0153151 0.0095426 -1.605 0.11973
## I(CBMF * WEIGHT) 0.0090461 0.0097924 0.924 0.36349
## I(ALMC * WEIGHT) -0.0042285 0.0102456 -0.413 0.68295
## I(RKXMIN * CVSG) 0.0135809 0.0099506 1.365 0.18317
## I(CBMF * CVSG) -0.0028367 0.0085528 -0.332 0.74261
## I(ALMC * CVSG) 0.0009064 0.0099510 0.091 0.92807
## I(CBMF * RKXMIN) -0.0221140 0.0107326 -2.060 0.04876 *
## I(ALMC * RKXMIN) -0.0119784 0.0095835 -1.250 0.22168
## I(ALMC * CBMF) -0.0185742 0.0087579 -2.121 0.04292 *
## I(sin(pi * CVSG * 1)) -0.0122984 0.0068611 -1.792 0.08387 .
## I(cos(pi * CVSG * 1)) -0.0183171 0.0040402 -4.534 9.91e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.01834 on 28 degrees of freedom
## Multiple R-squared: 0.9927, Adjusted R-squared: 0.9847
## F-statistic: 123.1 on 31 and 28 DF, p-value: < 2.2e-16
##
## [1] "1 I(ALMC * CVSG)"
## [1] "2 I(CBMF * WEIGHT)"
## [1] "3 I(CVSG * UICEFAC)"
## [1] "4 I(CBMF * CVSG)"
## [1] "5 I(RKXMIN * WEIGHT)"
##
## Call:
## lm(formula = PCP ~ CT + UICEFAC + I(UICEFAC^2) + WEIGHT + CVSG +
## RKXMIN + CBMF + ALMC + I(UICEFAC * CT) + I(WEIGHT * CT) +
## I(CVSG * CT) + I(RKXMIN * CT) + I(CBMF * CT) + I(ALMC * CT) +
## I(WEIGHT * UICEFAC) + I(RKXMIN * UICEFAC) + I(CBMF * UICEFAC) +
## I(ALMC * UICEFAC) + I(CVSG * WEIGHT) + I(ALMC * WEIGHT) +
## I(RKXMIN * CVSG) + I(CBMF * RKXMIN) + I(ALMC * RKXMIN) +
## I(ALMC * CBMF) + I(sin(pi * CVSG * 1)) + I(cos(pi * CVSG *
## 1)), data = tData)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.027609 -0.011052 -0.000731 0.009767 0.036193
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.873861 0.004230 679.430 < 2e-16 ***
## CT 0.188658 0.005000 37.733 < 2e-16 ***
## UICEFAC 0.140343 0.004781 29.357 < 2e-16 ***
## I(UICEFAC^2) -0.048240 0.009730 -4.958 2.09e-05 ***
## WEIGHT -0.064749 0.005371 -12.055 1.24e-13 ***
## CVSG 0.047045 0.008420 5.587 3.26e-06 ***
## RKXMIN -0.025270 0.004663 -5.420 5.34e-06 ***
## CBMF 0.009825 0.005078 1.935 0.061626 .
## ALMC -0.009339 0.004766 -1.960 0.058519 .
## I(UICEFAC * CT) 0.041193 0.011651 3.536 0.001230 **
## I(WEIGHT * CT) 0.029289 0.012831 2.283 0.029021 *
## I(CVSG * CT) -0.021481 0.009709 -2.212 0.033964 *
## I(RKXMIN * CT) 0.007921 0.008340 0.950 0.349141
## I(CBMF * CT) 0.007012 0.009239 0.759 0.453251
## I(ALMC * CT) -0.018255 0.009467 -1.928 0.062453 .
## I(WEIGHT * UICEFAC) -0.006205 0.008806 -0.705 0.485955
## I(RKXMIN * UICEFAC) -0.006853 0.008911 -0.769 0.447386
## I(CBMF * UICEFAC) 0.021240 0.009046 2.348 0.025025 *
## I(ALMC * UICEFAC) -0.011160 0.008044 -1.387 0.174629
## I(CVSG * WEIGHT) -0.013657 0.008478 -1.611 0.116735
## I(ALMC * WEIGHT) -0.004746 0.009337 -0.508 0.614618
## I(RKXMIN * CVSG) 0.013103 0.009824 1.334 0.191420
## I(CBMF * RKXMIN) -0.014998 0.009851 -1.522 0.137421
## I(ALMC * RKXMIN) -0.015136 0.009350 -1.619 0.115018
## I(ALMC * CBMF) -0.017732 0.008709 -2.036 0.049838 *
## I(sin(pi * CVSG * 1)) -0.010184 0.006501 -1.566 0.126800
## I(cos(pi * CVSG * 1)) -0.016754 0.003898 -4.298 0.000143 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.01842 on 33 degrees of freedom
## Multiple R-squared: 0.9913, Adjusted R-squared: 0.9845
## F-statistic: 145.3 on 26 and 33 DF, p-value: < 2.2e-16
##
## [1] "1 I(RKXMIN * UICEFAC)"
## [1] "2 I(RKXMIN * CVSG)"
## [1] "3 I(ALMC * RKXMIN)"
## [1] "4 I(CBMF * CT)"
## [1] "5 I(CBMF * RKXMIN)"
## [1] "6 I(ALMC * UICEFAC)"
##
## Call:
## lm(formula = PCP ~ CT + UICEFAC + I(UICEFAC^2) + WEIGHT + CVSG +
## RKXMIN + CBMF + ALMC + I(UICEFAC * CT) + I(WEIGHT * CT) +
## I(CVSG * CT) + I(RKXMIN * CT) + I(ALMC * CT) + I(WEIGHT *
## UICEFAC) + I(CBMF * UICEFAC) + I(CVSG * WEIGHT) + I(ALMC *
## WEIGHT) + I(ALMC * CBMF) + I(sin(pi * CVSG * 1)) + I(cos(pi *
## CVSG * 1)), data = tData)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.028266 -0.009710 -0.001087 0.012051 0.036737
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.877182 0.003907 736.395 < 2e-16 ***
## CT 0.189534 0.004792 39.554 < 2e-16 ***
## UICEFAC 0.140620 0.004768 29.494 < 2e-16 ***
## I(UICEFAC^2) -0.056605 0.009168 -6.174 2.97e-07 ***
## WEIGHT -0.065363 0.004998 -13.078 7.72e-16 ***
## CVSG 0.042599 0.007918 5.380 3.75e-06 ***
## RKXMIN -0.024057 0.004665 -5.157 7.63e-06 ***
## CBMF 0.013363 0.004857 2.751 0.008958 **
## ALMC -0.009116 0.004709 -1.936 0.060160 .
## I(UICEFAC * CT) 0.031147 0.010628 2.931 0.005629 **
## I(WEIGHT * CT) 0.028628 0.011933 2.399 0.021319 *
## I(CVSG * CT) -0.020274 0.009710 -2.088 0.043375 *
## I(RKXMIN * CT) 0.004727 0.008228 0.575 0.568926
## I(ALMC * CT) -0.023011 0.009046 -2.544 0.015045 *
## I(WEIGHT * UICEFAC) -0.009366 0.008443 -1.109 0.274046
## I(CBMF * UICEFAC) 0.025593 0.008356 3.063 0.003962 **
## I(CVSG * WEIGHT) -0.014917 0.008298 -1.798 0.079966 .
## I(ALMC * WEIGHT) -0.011536 0.008951 -1.289 0.205029
## I(ALMC * CBMF) -0.016532 0.008257 -2.002 0.052251 .
## I(sin(pi * CVSG * 1)) -0.008557 0.006341 -1.349 0.184960
## I(cos(pi * CVSG * 1)) -0.015562 0.003893 -3.997 0.000276 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.0189 on 39 degrees of freedom
## Multiple R-squared: 0.9892, Adjusted R-squared: 0.9837
## F-statistic: 179 on 20 and 39 DF, p-value: < 2.2e-16DW to edit this function to remove requirement for canfacs and to explain output object.
names(myem.lm.canPCP)## [1] "Names" "linModel" "mainEffects"
## [4] "Interactions" "Factors" "FactorInteractions"
## [7] "ThreeWayInters" "Fouriers" "pre.Lists"
## [10] "DataString" "ResponseString"summary(myem.lm.canPCP$linModel)##
## Call:
## lm(formula = PCP ~ CT + UICEFAC + I(UICEFAC^2) + WEIGHT + CVSG +
## RKXMIN + CBMF + ALMC + I(UICEFAC * CT) + I(WEIGHT * CT) +
## I(CVSG * CT) + I(RKXMIN * CT) + I(ALMC * CT) + I(WEIGHT *
## UICEFAC) + I(CBMF * UICEFAC) + I(CVSG * WEIGHT) + I(ALMC *
## WEIGHT) + I(ALMC * CBMF) + I(sin(pi * CVSG * 1)) + I(cos(pi *
## CVSG * 1)), data = tData)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.028266 -0.009710 -0.001087 0.012051 0.036737
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.877182 0.003907 736.395 < 2e-16 ***
## CT 0.189534 0.004792 39.554 < 2e-16 ***
## UICEFAC 0.140620 0.004768 29.494 < 2e-16 ***
## I(UICEFAC^2) -0.056605 0.009168 -6.174 2.97e-07 ***
## WEIGHT -0.065363 0.004998 -13.078 7.72e-16 ***
## CVSG 0.042599 0.007918 5.380 3.75e-06 ***
## RKXMIN -0.024057 0.004665 -5.157 7.63e-06 ***
## CBMF 0.013363 0.004857 2.751 0.008958 **
## ALMC -0.009116 0.004709 -1.936 0.060160 .
## I(UICEFAC * CT) 0.031147 0.010628 2.931 0.005629 **
## I(WEIGHT * CT) 0.028628 0.011933 2.399 0.021319 *
## I(CVSG * CT) -0.020274 0.009710 -2.088 0.043375 *
## I(RKXMIN * CT) 0.004727 0.008228 0.575 0.568926
## I(ALMC * CT) -0.023011 0.009046 -2.544 0.015045 *
## I(WEIGHT * UICEFAC) -0.009366 0.008443 -1.109 0.274046
## I(CBMF * UICEFAC) 0.025593 0.008356 3.063 0.003962 **
## I(CVSG * WEIGHT) -0.014917 0.008298 -1.798 0.079966 .
## I(ALMC * WEIGHT) -0.011536 0.008951 -1.289 0.205029
## I(ALMC * CBMF) -0.016532 0.008257 -2.002 0.052251 .
## I(sin(pi * CVSG * 1)) -0.008557 0.006341 -1.349 0.184960
## I(cos(pi * CVSG * 1)) -0.015562 0.003893 -3.997 0.000276 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.0189 on 39 degrees of freedom
## Multiple R-squared: 0.9892, Adjusted R-squared: 0.9837
## F-statistic: 179 on 20 and 39 DF, p-value: < 2.2e-16par(mfrow=c(2,2),mar=c(4,4,1,1))
plot(myem.lm.canPCP$linModel)
Are all of the items in this list required by the core emulator code? What is required? Can the object be simplified? Should it be? How will we extract what we require from an alternative mean function?
Build the emulator object
An emulator requires a mean function, a form for the covariance function, a method, and priors for parameters. The Emulator object contains everything that we can pre-compute that is required for prediction. This includes, for our fully Bayesian emulators, a set of posterior samples for the parameters. The EMULATE functions build emulators. Below is an example using the gpstan method (Gaussian process fitted in Stan). - The mean function is our linear model. - Currently we specify a nugget (and there are alternative methods coded up), we need to remove or default this requirement. - We pass the tData object. - additionalVariables asks which inputs, in addition to those fitted in the meanResponse, are “Active” - CreatePredictObject was an attempt to compile the stan file for prediction, but this didn’t work - FastVersion builds all of the required covariance matrices/cholesky factors at the MAP estimate of the parameters so that fast prediction and diagnostics are possible.
CanadaEmulator <- EMULATE.gpstan(meanResponse=myem.lm.canPCP, nugget=0.0001, tData=tData, additionalVariables=NULL, CreatePredictObject = FALSE, FastVersion = TRUE)## In file included from filee4974f5c2640.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: 'BOOST_NO_CXX11_RVALUE_REFERENCES' macro redefined [-Wmacro-redefined]
## # define BOOST_NO_CXX11_RVALUE_REFERENCES
## ^
## <command line>:5:9: note: previous definition is here
## #define BOOST_NO_CXX11_RVALUE_REFERENCES 1
## ^
## In file included from filee4974f5c2640.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:42:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:17: warning: unused function 'set_zero_all_adjoints' [-Wunused-function]
## static void set_zero_all_adjoints() {
## ^
## In file included from filee4974f5c2640.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:43:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints_nested.hpp:17:17: warning: 'static' function 'set_zero_all_adjoints_nested' declared in header file should be declared 'static inline' [-Wunneeded-internal-declaration]
## static void set_zero_all_adjoints_nested() {
## ^
## In file included from filee4974f5c2640.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:58:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/autocorrelation.hpp:17:14: warning: function 'fft_next_good_size' is not needed and will not be emitted [-Wunneeded-internal-declaration]
## size_t fft_next_good_size(size_t N) {
## ^
## In file included from filee4974f5c2640.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:298:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr.hpp:38:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr/functor/integrate_ode_rk45.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint.hpp:61:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint/util/multi_array_adaption.hpp:29:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array.hpp:21:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/base.hpp:28:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:43:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:53:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:54:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## 8 warnings generated.To predict with an emulator requires a predict-type function. In our code, the EMULATOR functions call emulators (for prediction/diagnostics or otherwise). Here we call a gpstan emulator.
validation <- EMULATOR.gpstan(x=tData[1:10,], Emulator=CanadaEmulator)## In file included from filee49768f47ae.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: 'BOOST_NO_CXX11_RVALUE_REFERENCES' macro redefined [-Wmacro-redefined]
## # define BOOST_NO_CXX11_RVALUE_REFERENCES
## ^
## <command line>:5:9: note: previous definition is here
## #define BOOST_NO_CXX11_RVALUE_REFERENCES 1
## ^
## In file included from filee49768f47ae.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:42:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:17: warning: unused function 'set_zero_all_adjoints' [-Wunused-function]
## static void set_zero_all_adjoints() {
## ^
## In file included from filee49768f47ae.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:43:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints_nested.hpp:17:17: warning: 'static' function 'set_zero_all_adjoints_nested' declared in header file should be declared 'static inline' [-Wunneeded-internal-declaration]
## static void set_zero_all_adjoints_nested() {
## ^
## In file included from filee49768f47ae.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:58:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/autocorrelation.hpp:17:14: warning: function 'fft_next_good_size' is not needed and will not be emitted [-Wunneeded-internal-declaration]
## size_t fft_next_good_size(size_t N) {
## ^
## In file included from filee49768f47ae.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:298:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr.hpp:38:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr/functor/integrate_ode_rk45.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint.hpp:61:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint/util/multi_array_adaption.hpp:29:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array.hpp:21:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/base.hpp:28:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:43:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:53:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:54:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## 8 warnings generated.
##
## SAMPLING FOR MODEL 'PredictGen' NOW (CHAIN 1).
## Iteration: 1 / 1 [100%] (Sampling)
##
## Elapsed Time: 0 seconds (Warm-up)
## 0.487758 seconds (Sampling)
## 0.487758 seconds (Total)validation## $Expectation
## [1] 2.856839 2.954878 3.141369 2.789831 2.931018 2.688995 2.881553
## [8] 2.915374 3.044184 2.620686
##
## $Variance
## [1] 0.0001731364 0.0001811062 0.0001863176 0.0001834364 0.0001776451
## [6] 0.0001902292 0.0001846268 0.0001929097 0.0001837244 0.0001841843Sitting inside our history matching functions are calls to EMULATOR objects (see UniImplausibility, for example).
Speed
One of our main issues is speed for files calling Stan. E.g.
tLOOs <- stanLOOplot(StanEmulator = CanadaEmulator, ParamNames=CanadaEmulator$Names)## In file included from filee49734cc72b1.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: 'BOOST_NO_CXX11_RVALUE_REFERENCES' macro redefined [-Wmacro-redefined]
## # define BOOST_NO_CXX11_RVALUE_REFERENCES
## ^
## <command line>:5:9: note: previous definition is here
## #define BOOST_NO_CXX11_RVALUE_REFERENCES 1
## ^
## In file included from filee49734cc72b1.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:42:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:17: warning: unused function 'set_zero_all_adjoints' [-Wunused-function]
## static void set_zero_all_adjoints() {
## ^
## In file included from filee49734cc72b1.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:43:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints_nested.hpp:17:17: warning: 'static' function 'set_zero_all_adjoints_nested' declared in header file should be declared 'static inline' [-Wunneeded-internal-declaration]
## static void set_zero_all_adjoints_nested() {
## ^
## In file included from filee49734cc72b1.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:58:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/autocorrelation.hpp:17:14: warning: function 'fft_next_good_size' is not needed and will not be emitted [-Wunneeded-internal-declaration]
## size_t fft_next_good_size(size_t N) {
## ^
## In file included from filee49734cc72b1.cpp:8:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:12:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat.hpp:298:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr.hpp:38:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr/functor/integrate_ode_rk45.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint.hpp:61:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/numeric/odeint/util/multi_array_adaption.hpp:29:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array.hpp:21:
## In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/base.hpp:28:
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:43:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:53:43: warning: unused typedef 'index_range' [-Wunused-local-typedef]
## typedef typename Array::index_range index_range;
## ^
## /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:54:37: warning: unused typedef 'index' [-Wunused-local-typedef]
## typedef typename Array::index index;
## ^
## 8 warnings generated.
##
## SAMPLING FOR MODEL 'PredictLOO' NOW (CHAIN 1).
## Iteration: 1 / 1 [100%] (Sampling)
##
## Elapsed Time: 0 seconds (Warm-up)
## 25.2457 seconds (Sampling)
## 25.2457 seconds (Total)
takes a very long time (relatively) to run. Run time seems to increase exponentially with ensemble size, and this is one of the main issues we have. Mainly this is a problem with Stan, but it will require our writing the code such that large ensembles are never passed to Stan without the user having to do an awful lot of working around our default (safety?) settings.