Module 11: Repeated measures I, simple methods

Summary of experiment:

• 3 treatments: 1, 2, 3 (concentration)
• 10 cages per treatment
• 10 contiguous months
• The response is activity (count of intersections of light beam during 57 hours).

One way to avoid the problem of correlated measurements is to do a separate analysis for each point in time. This way only one observation from each individual is used, and hence they are independent. This way of analyzing repeated measurements is not wrong, but it is very inefficient, as all the remaining observations are waisted. This approach avoids the problem, instead of dealing with it.

Separate analysis can be carried out for all the observed time-points, but it will likely be very difficult to reach a coherent conclusion from all these sub-tests. These sub-tests will be correlated, and because the correlation structure is not part of the model, it is not possible to tell how strong this correlation is.

Separate analysis can be carried out for selected time-points ``far apart''. This will (hopefully) cause the separate sub-tests to be uncorrelated, or at least less correlated. Even with uncorrelated tests it will be difficult to reach a coherent conclusion, because of a problem known as mass significance (or multiplicity) . For instance, if 20 tests are carried out at a 5% significance level, one of them is likely to be false significant. This problem is partly solved by using the Bonferroni correction for performing n tests (one for each time-point). The Bonferroni correction simply states that the P-value 0.05/n should be used instead of the usual 0.05.

When selecting time-points far apart, is important that the selection must be done independently of the actual observations. Naturally the time-points may not be selected systematically where there is large (or small) difference between treatments. Ideally the time points should be selected before data is collected.

Example: Activity of rats analyzed separately for each month

For the analysis in this section it is assumed that the data is read into a SAS data set named rats with the columns treatm (=1,2,3), cage (=cage number), month (=1¼10), and lnc (=log(counts)). The data set has 300 lines. Here is the fist few lines:

Obs    treatm    cage    month      lnc
1       1        1        1      9.9323
2       1        1        2      9.6447
3       1        1        3      9.7628
4       1        1        4      9.6014
5       1        1        5      9.3227
6       1        1        6      9.2463
.       .        .        .       .
.       .        .        .       .
.       .        .        .       .

To analyze the rats data set separately for each month, a simple one way analysis of variance model with treatment treatm as the only factor is used. The information about cage can not be included, as we only have one observation from each cage in each monthly analysis. The model for each month is:

lnci = m+ a(treatmi)+ei ,    ei ~ i.i.d. N(0,s2),     i=1¼30

To analyze this via proc mixed in SAS, write:

proc sort data=rats; by month; run;
proc mixed data=rats method=REML;
  class treatm month;
  model lnc = treatm;
  by month;
run;

The SAS output is summarized in the following table of F-tests for no treatment effect:

A few significant values are found, and even one where the Bonferroni correction is used, so the conclusion should be that weak evidence of group difference have been seen.

It is possible to make a correct analysis time-by-time, but it is weak and often confusing, because it does not combine all information into one test.

Another way to avoid the problem of correlated measurements is to choose a single measure to summarize the individual curves, and then base the analysis on this measure. This again reduces the data set to independent ``observations'' - one for each individual. To analyze the summary data set, standard methods for independent observations for instance analysis of variance can be used.

The key is to choose a good summary measure. One possibility is to choose the value at a given time-point, which reduces this summary method to the separate time-point analysis described in the previous section. This choice is poor in most cases, because all other measurements are waisted.

It is difficult to give general advice about the choice of summery measurement. Ideally, the summary measure should capture the most important feature of the curve. In some situations the most important feature is the net growth (last minus first), the average growth (slope), or time to reach the maximum point. It depends on the problem at hand.

Some common choices of summary measures are:

• Average over time
• Slope in regression with time (or higher order polynomial coefficients)
• Total increase (last point minus first point)
• Area under curve (AUC)
• Maximum or minimum point

With the right choice of summary measure this type of analysis can be very useful, at least as a first step. These models have relatively few assumptions, and they can be checked via standard residual methods. Of course the downside of this method is that information may be lost by reducing each curve to one single measure.

Example: Activity of rats analyzed via summary measure

The two approaches described above both illustrated ways to reduce the data set to independent measures. This section explains the first step in modeling the actual covariance structure in the data set.

As seen in previous modules, for instance the module about hierarchial random effects, the effect of adding a random effect is that two observations from the same level will possibly be positively correlated. Adding the ``individual'' factor to the model as a random effect will allow two observations from the same individual to be positively correlated.

Example: Activity of rats analyzed via random effects model

It is reasonable to assume that two observations from the same cage could be correlated, so the model with cage as random effect is used. The factor month and the interaction between month and treatment are included. This was not possible in the previous models, because each curve was reduced into one number. In this analysis all observations are included into one coherent analysis. The model is:

lnci = m+a(treatmi) +b(monthi) +g(treatmi,monthi) +d(cagei) +ei ,

where i=1¼300, d(cage_i) ~ N(0,s²_d), e_i ~ N(0,s²), and all independent.

Recall from previous modules that the covariance structure for this model is:

cov(yi1, yi2) = ì ï ï í ï ï î 0 , if cagei1 ¹ cagei2 and i1 ¹ i2 sd2 , if cagei1 = cagei2 and i1 ¹ i2 sd2+s2 , if i1 = i2

In other words this is the variance structure, where two observations from different cages are uncorrelated, and two observations from the same cage are positively correlated with correlation coefficient s_d²/(s_d²+s²).

The following lines implement this model in SAS:

proc mixed data=rats;
  class treatm cage month;
  model lnc = treatm month treatm*month / ddfm=satterth;
  random cage;
run;

Covariance Parameter Estimates
Cov Parm     Estimate
cage          0.02748
Residual      0.03790

-2 Res Log Likelihood      8.6

                 Num     Den
Effect            DF      DF    F Value    Pr > F
treatm             2      27       3.22    0.0557
month              9     243      46.11    <.0001
treatm*month      18     243       2.12    0.0059

This output give estimates of the variance parameters (s²_d=0.02748 and s²=0.03790), twice the negative restricted/residual log likelihood (2l_re=8.6), and an ANOVA table for the fixed effects of the model. From this ANOVA table it is seen that the interaction between treatment and month is significant with a P-value=0.0059. The conclusion from this model is that treatment does have an effect on the activity, but the effect is not the same in all ten months.

The main problem with this random effects approach is that all measurements on the same individual are assumed equally correlated, but some measurements are taken far apart and some measurements are taken close to each other, so this assumption is not always valid. The next module will suggest a few ways to deal with this problem.

However, this random effects approach may give reasonable results for short series (with 2, 3, or 4 measurements on each individual) since the assumption of equal correlation may be ok in those cases.

This random effects approach is also known as the split-plot approach, or the split-plot model. It is possible to view repeated measurements data as resulting from a kind of split-plot experiment, with individuals as the ``main-plots'' to which the the treatments are applied. The ``sub-plots'' are then the single measurements on each individuals. This interpretation is a bit weak, as the single measurements on each individual (typically at different times) cannot be randomized within the individual.

In this module a few simple approaches to the analysis of repeated measurements have been described. In many practical cases these simple approaches, especially the summary method, will give a sufficient and useful analysis of the data. Even in those cases where more sophisticated models are needed it is often helpful to run a few simple models first. Here follows a few pros and cons of the different methods:

Separate analysis for each time-point

+ Not wrong
-- Can be confusing
-- Difficult to reach coherent conclusion
-- In general not very informative

Analysis of summary statistic

+ Good method with few and easily checked assumptions
-- Important to choose good summary measure(s)

Random effects approach

+ Good method for short series
+ Uses all observations
-- Usually not good for long series