Is that correlation of PPG on in sample or out of sample data? The number you cited of 1400 leads me to think it’s in sample performance but I could be wrong. Also you’re using a Bayesian regression to quantify uncertainty but I don’t see any calibration metrics for your posterior predictive samples. Also is the conference the only random effect on your hierarchical model?
Hi there. The correlation reflects in-sample performance on the 2025-26 training data, so I added a 20% holdout withheld from training entirely to get an unbiased out-of-sample evaluation. The model does include posterior predictive calibration, I just kept the write-up somewhat high level to avoid getting too technical and confusing people. As for random effects, the model has several hierarchical factors, all of which sample without any convergence issues.
Is that correlation of PPG on in sample or out of sample data? The number you cited of 1400 leads me to think it’s in sample performance but I could be wrong. Also you’re using a Bayesian regression to quantify uncertainty but I don’t see any calibration metrics for your posterior predictive samples. Also is the conference the only random effect on your hierarchical model?
Hi there. The correlation reflects in-sample performance on the 2025-26 training data, so I added a 20% holdout withheld from training entirely to get an unbiased out-of-sample evaluation. The model does include posterior predictive calibration, I just kept the write-up somewhat high level to avoid getting too technical and confusing people. As for random effects, the model has several hierarchical factors, all of which sample without any convergence issues.