Bayesian methods engine pack covering core concepts (priors, posteriors, credible intervals, conjugacy), inference algorithms (MCMC, HMC, NUTS, variational inference, ABC), hierarchical/multilevel models, and model checking (posterior predictive checks, Bayes factors, effective sample size). Bundles authoritative references from Gelfand & Smith (1990) through Stan (Carpenter et al. 2017) and the modern Bayesian workflow (van de Schoot et al. 2021). Includes a playbook for the killer demo: using KB findings from other packs as informative priors in Bayesian regression.
Gibbs sampling and related sampling-based approaches provide a general, computationally feasible route to marginal and conditional posterior distributions in Bayesian hierarchical models where analytical integration is intractable. Enabled the wide adoption of MCMC for applied Bayesian inference in the 1990s.
Inverse-Gamma(ε, ε) priors on variance parameters — a historically common 'non-informative' default in hierarchical Bayesian models — are in fact strongly informative with influence that does not vanish as ε → 0. Half-Cauchy and half-Normal priors on the standard deviation are recommended weakly-informative alternatives that avoid this pathology.
The No-U-Turn Sampler matches or exceeds the efficiency of hand-tuned HMC across a range of target distributions while eliminating the need to specify trajectory length, and outperforms random-walk Metropolis by one to three orders of magnitude in effective sample size per gradient evaluation on correlated Gaussian and logistic regression targets.
Domain: Bayesian Methods
Bayesian statistical methods — priors and posteriors, conjugate families, credible intervals, MCMC/HMC/NUTS/variational inference, hierarchical and multilevel models, posterior predictive checks, Bayes factors, and the modern Bayesian workflow. Engine pack focus: turning KB findings from other packs into informative priors for Bayesian regression.
…and 4 more findings
Gibbs sampling and related sampling-based approaches provide a general, computationally feasible route to marginal and conditional posterior distributions in Bayesian hierarchical models where analytical integration is intractable. Enabled the wide adoption of MCMC for applied Bayesian inference in the 1990s.
Inverse-Gamma(ε, ε) priors on variance parameters — a historically common 'non-informative' default in hierarchical Bayesian models — are in fact strongly informative with influence that does not vanish as ε → 0. Half-Cauchy and half-Normal priors on the standard deviation are recommended weakly-informative alternatives that avoid this pathology.
The No-U-Turn Sampler matches or exceeds the efficiency of hand-tuned HMC across a range of target distributions while eliminating the need to specify trajectory length, and outperforms random-walk Metropolis by one to three orders of magnitude in effective sample size per gradient evaluation on correlated Gaussian and logistic regression targets.
Mean-field variational inference provides dramatically faster approximate posterior inference than MCMC on large-scale latent-variable models (e.g. LDA topic models with millions of documents), at the cost of underestimating posterior variance due to the KL(q||p) forward-KL objective that is mode-seeking rather than mean-matching.
Partial pooling via hierarchical models produces group-level estimates with lower mean squared error than either complete pooling (ignoring group structure) or no pooling (fitting separate models per group), particularly for groups with small sample sizes. The shrinkage is automatic and adapts to the estimated between-group variance.
Posterior predictive checks using realized discrepancy measures T(y, θ) — test quantities that may depend on both data and parameters — provide a principled Bayesian analogue to goodness-of-fit testing. Systematic divergence between T applied to observed vs. replicated data reveals specific modes of model misspecification that point estimates of fit cannot.
Bayes factors are sensitive to the choice of prior on model parameters in a way that p-values are not; improper priors under the models being compared render the Bayes factor undefined. Proper, weakly-informative priors chosen with the comparison in mind are required for interpretable model comparison.
Penalised Complexity (PC) priors constructed by penalizing KL divergence from a simpler base model (e.g. zero variance, independence) produce interpretable, weakly-informative defaults for random-effect variances, autoregressive correlation parameters, and overdispersion terms — with a single user-chosen scale that has clear probabilistic meaning.
Stan's implementation of NUTS with dynamic trajectory doubling, dual-averaging step-size adaptation, and dense-mass-matrix adaptation produces reliable posterior samples for a wide class of continuous-parameter Bayesian models without user tuning. Combined with automatic differentiation and a C++ backend, it enables Bayesian inference at scales previously requiring bespoke Gibbs samplers.
Approximate Bayesian Computation enables posterior inference for models with intractable likelihoods (e.g. coalescent simulators, agent-based models) by replacing likelihood evaluation with simulation and distance-based acceptance of parameters. ABC-SMC and regression-adjustment variants substantially improve efficiency over rejection ABC for moderate-dimensional parameter spaces.
The modern applied Bayesian workflow centers on iterative model building: start with weakly-informative priors and a simple model, use HMC/NUTS for posterior computation, apply posterior predictive checks to detect misspecification, refine priors and model structure, and validate via cross-validation (e.g. PSIS-LOO). Prior sensitivity analysis — refitting with varied priors — is recommended for any published Bayesian analysis.
Bayesian credible intervals admit the direct probabilistic interpretation that practitioners often incorrectly ascribe to frequentist confidence intervals. This interpretive clarity, combined with modeling flexibility for hierarchical and non-standard likelihoods, is a primary reason behavioral and social sciences have increasingly adopted Bayesian methods.
name: bayesian-methods version: 1.1.0 pax_type: engine author: Josh Lambert license: CC-BY-4.0 published_by: Praxis Agent domain: bayesian_methods constructs: - bayesian_linear_regression - conjugate_prior - informative_prior - weakly_informative_prior - posterior_distribution - credible_interval - markov_chain_monte_carlo - hamiltonian_monte_carlo - no_u_turn_sampler - variational_inference - hierarchical_model - posterior_predictive_check - bayes_factor - effective_sample_size engines: - bayesian_linear_regression counts: constructs: 14 findings: 12 propositions: 0 playbooks: 1 sources: 13