Bayz Manual

Basic Bayesian mixed linear models

On this page are examples for basic linear and linear mixed models: fit of factors and covariates in univariate models with uncorrelated residuals and random effects. Where variance are estimated the examples use uniform prior distributions, but in the section "Options and priors for variances" also other priors are explained.

Directly jump to:

Mean (intercept) term

Residual variance specification

Factor (class) variables, fixed and random

Regression on continuous covariates, fixed and random

Options and priors for variances

Mean term

Nearly all models will need an intercept, except for some special cases (e.g., hierarchical models). Bayz does not automatically add a model intercept. The intercept is added with the model term 'mean'. Typically the model-mean is a 'fixed' effect, which in Bayesian models has the equivalent of assigning a uniform prior. The following examples specifies to model weight as response variable with a model that only includes a mean (intercept) fitted 'fixed':

    model
    weight = mean
    mean.weight ~ uni
A full example including estimation of residual variance follows below.

Residual variance model

The most common assumption for the residuals is that they are uncorrelated and have the same variance (iid). Bayz also allows for weighted residual variance, and heterogeneous and across-trait correlated residual variances (explained elsewhere). Typically the variance is also estimated from the data, and the Bayesian model needs a prior distribution for the residual variance in order to estimate it. The following is a complete example with the age-weight data that uses the model with a mean only, specifies residuals as iid by using the 'iden' (identity) normal distribution for residuals, and estimates residual variance using a uniform prior on the variance (see further below to use other priors for the variance):

    data
    file=ageweight.txt
    mouseID sex age weight

    model
    weight = mean
    mean.weight ~ uni
    resid.weight ~ norm iden *
    var.resid.weight ~ uni

Factor fit

In other statistical packages this is also called a class variable, factor is borrowed from R where you would use factor(variable) in a linear model to fit an effect for every distinct level in a explanatory variable. In bayz, factor variables may be text, e.g., "M" and "F" for sex, a diet variable labelled "Control", "Protein" and "Carbo", etc.

Fixed effects factor

The following shows how the mean-model is extended to add a factor sex as explanatory variable, and uses as uniform prior on the sex effects to make the Bayesian equivalent of a 'fixed effect':

    weight = mean fac.sex
    mean.weight ~ uni
    fac.sex.weight ~ uni

Random effects factor

A factor can be switched from 'fixed effect' to 'random effect' by switching the prior uniform distribution to a normal distribution. The simplest version has random effects that are uncorrelated and identically distributed, which is the 'norm iden' (Identity) distribution with one variance component. The following is an example that could be used when repeated observations are present, fitting an animal ID as random effect to capture the animal repeatability. The variance (var.fac.mouseID.weight) is assigned a uniform prior in this example (see further below to use other priors for the variance).

    weight = mean fac.sex fac.mouseID
    mean.weight ~ uni
    fac.sex.weight ~ uni
    fac.mouseID.weight ~ norm iden *
    var.fac.mouseID.weight ~ uni

Regression (covariate) fit

Simple fixed effects regression

To fit a regression on a continuous covariate, see the example below that extends the model for weight with a regression on age. A uniform prior for the regression is set to fit the regression as 'fixed'.

    weight = mean fac.sex reg.age
    mean.weight ~ uni
    fac.sex.weight ~ uni
    reg.age.weight ~ uni

Nested fixed effects regression

A nested regression, for instance regression on age within sexes, and setting a uniform prior to model these regression coefficients 'fixed':

    weight = mean fac.sex reg.age:sex
    mean.weight ~ uni
    fac.sex.weight ~ uni
    reg.age:sex.weight ~ uni

Random effects regression

Below is an example with random nested regressions. Random regression are also typical on blocks of covariates (see elsewhere). The example below could be for repeated weight data, modelling random mouse effects, a base-line fixed regression of age on weight, and random nested regressions for each mouse on weight. It could be useful to extend the model to allow for correlation between the mouse and mouse-age effects - see elsewhere. The random regressions use the basic 'norm iden' (uncorrelated) variance structure, an alternative would be to use pedigree or genomic information to correlate the mouse effect - also see elsewhere.

    weight = mean fac.sex reg.age fac.mouseID reg.age:mouseID
    mean.weight ~ uni
    fac.sex.weight ~ uni
    reg.age.weight ~ uni
    fac.mouseID.weight ~ norm iden *
    reg.age:mouseID.weight ~ norm iden *
    var.fac.mouseID.weight ~ uni
    var.reg.age:mouseID.weight ~ uni

Options and priors for variances

It is possible to modify the starting value for a variance component (often not so crucial), to set whether a variance should be known or estimated in the model, and if the variance is estimated, what prior distribution to use, see the following examples (all will work with residual variance, variances for random effects and for variances of random regressions). These examples use the 'norm iden' (identity) variance structure, most things also apply to other variance structures.

Variance estimated with default starting value 1:

    resid.weight ~ norm iden *

Variance estimated with starting value 8.5:

    fac.mouseID.weight ~ norm iden *8.5

Variance not estimated and set known at the value 9.71 (note the missing asterix):

    fac.mouseID.weight ~ norm iden 9.71

For an estimated variance, specification of a uniform prior:

    var.resid.weight ~ uni

For an estimated variance, specification of a scale-inverse chi-square prior with scale 6.5 and degrees of freedom 3:

    var.fac.mouseID.weight ~ ichi 6.5 3

The scale of the scale-inverse chi-square distribution can also be estimated, see elsewhere.

Variance with an exponential prior, used for instance in the Bayesian LASSO model, typically applied to sets of regression coefficients where the rate of the exponential distribution is also estimated. The rate parameter can have a uniform or gamma distribution (here shown for a gamma):

    var.reg.metabol.weight ~ exp *20
    rate.var.reg.metabol.weight ~ gamm 5 10