General Linear Model (GLM)
General Linear Model (GLM)
normally distributed dependent variables and combinations of categorical and continuous independent (predictor) variables. GLM procedure can accommodate univariate models (one dependent variable) involving: a) Categorical predictors only (ANOVA) b) Continuous predictors only (Regression) c) Combinations of categorical predictors and continuous predictors (ANCOVA) It also can accommodate two or more dependent variables of all the above models (multivariate models). Categorical predictors are referred to as factors, while continuous predictors are called covariates. GLM can also fit repeated measures or within-subjects models, including doubly multivariate repeated measures models involving multiple measures at each time point or under each combination of conditions. To use a GLM procedure, from the menus choose one of the following: Analyze General Linear Model Univariate or Multivariate or Repeated Measures
ONE-WAY REPEATED MEASURES ANOVA In a one-way repeated measures ANOVA design each subject is exposed to two or more different conditions, or measured on the same continuous scale on three or more occasions. It can also be used to compare respondents responses to two or more different questions or items. These questions however must be measured using the same scale (eg. 1 = strongly agree, to 5 = strongly disagree). Example: A total of 100 patients who needed cataract operation and fulfilled the inclusion and exclusion criteria were selected from patients who were referred to HUKM. These patients were randomly allocated into two groups; 50 in ECCE and 50 in Phaco group. Effectiveness of cataract operation was assessed by disease specific quality of life score using the Visual Function 14 (VF-14) questionnaire. The score produced by this questionnaire range from 0 (unable to do all applicable activities) and a maximum of 100 (able to do all applicable items without difficulty). VF-14 questionnaire was administered prior to operation, one week, two months and six months after cataract operation. Details of the variables names and labels from the data file are as follows: File name CIS Variable name vf.preop Variable label VF-14 scores pre-operation VF-14 scores one week postoperation VF-14 scores two months post-operation VF-14 scores six months post-operation Coding instructions Total scores on the visual functioning administered prior to the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered one week after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered 2 months after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered 6 months after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning.
vf.week1
vf.mnth2
vf.mnth6
3 Research question: Is there a change in VF-14 scores over the four time periods for both groups of patients? This example involves two variables: a) One independent variable (categorical data) Time 1 (pre-op), Time 2 (one week), Time 3 (2 months) and Time 4 (6 months). b) One dependent variable (continuous data) scores on the VF-14 One-way repeated measures ANOVA will tell you if there is a significant difference somewhere among the four sets of scores. Assumptions: a) Normal distribution i) ii) iii) iv) v) vi) Kolmogorov-Smirnov statistic (a non-significant result, p>0.05 indicates normality) Skewness and kurtosis values (if the distribution is perfectly normal, the value of skewness and kurtosis are 0) Histograms (appear to be reasonably normally distributed) Normal Q-Q Plots (a reasonably straight line between observed value for each score against the expected value from the normal distribution, suggests a normal distribution) Detrended Normal Q-Q Plots (there should be no real clustering of points, with most collecting around the zero line) Boxplot (the rectangle represents 50% of the cases, with the whiskers (the lines protruding from the box) going out to the smallest and largest values. Additional circles outside the range is called outliers. The line inside the rectangle is the median value.
b) Homogeneity of variance The samples are assumed to be obtained from populations of equal variances. This means that the variability of scores for each of the groups is similar. Levenes test can be performed to test the equality of variances as part of the t-test and analysis of variances analyses. A significance value of less than 0.05, suggests that variances for the two groups are not equal, and therefore the assumption of homogeneity of variance is violated. Analysis of variance is reasonably robust to violations of this assumption, provided the size of the groups is reasonably similar (largest/smallest = 1.5). For t-tests, there are two sets of results provided, for equal variance assumed and not assumed. In this case, choose whichever set of results that is appropriate for your data.
PROCEDURE FOR ONE-WAY REPEATED MEASURES ANOVA 1. Open file CIS 2. From the menu, click: Analyze General Linear Model Repeated Measures 3. In the Within Subject Factor Name box type in a name that represents your independent variable (time). This is not an actual variable name, just a label you give your independent variable. 4. In the Number of Levels box type the number of levels or groups (time periods) involved (in this example it is 4).
5. Click Add. 6. Click on the Define button on the right-hand side. 7. Select the four variables that represent your repeated measures variable (vf.preop, vf.week1, vf.mnth2, vf.mnth6). Click on the arrow button to move them into the Within Subjects Variables box.
8. Click on the Options box at the bottom right of your screen. 9. Tick the Descriptive Statistics and Estimates of effect size boxes in the area labeled Display. 10. Click on Continue and then OK. The output generated from this procedure is shown below.
Descriptive Statistics Mean VF-14 score operation VF-14 score VF-14 score VF-14 score - pre - week 1 - month 2 - month 6 66.2124 90.6578 94.3863 96.2079 Std. Deviation 19.2653 10.1244 6.9793 6.0429 N 100 100 100 100
b M ultivariate Tests
Effect Value TIM E Pillai's T race .698 Wilks' Lam bda .302 Hotelling's Trace 2.311 Roy's Largest Root 2.311 a. Exact statistic b. Design: Intercept Within Subjects Design: T IM E
T e s ts th e n u ll h y p o th e s is th a t th e e r ro r c o v a ria n c e m a tr ix o f th e o r th o n o r m a liz e d tr a n s fo r m e d d e p e n d e n t v a r ia b le s p r o p o rtio n a l to a n id e n tity m a trix . a . M a y b e u s e d to a d ju s t th e d e g r e e s o f fre e d o m fo r th e a v e r a g e d te s ts o f s ig n ific a n c e . C o r r e c te d te s ts a re d is p l T e s ts o f W ith in -S u b je c ts E ffe c ts ta b le . b. D e s ig n : In te rc e p t W ith in S u b je c ts D e s ig n : T IM E
T e s ts o f W ith in -Su b je c ts E ffe c ts M e a su re : M E A SU R E_ 1 T yp e III Su m o f S q u a re s df M e a n Sq u a re Sp h e ric ity A ssu m e d 5 8 4 7 7 .4 3 8 3 1 9 4 9 2 .4 7 9 G re e n h o u se -G e isse r 5 8 4 7 7 .4 3 8 1 .7 0 0 3 4 4 0 3 .6 3 4 H u yn h -F e ld t 5 8 4 7 7 .4 3 8 1 .7 2 6 3 3 8 7 3 .3 9 4 L o w e r-b o u n d 5 8 4 7 7 .4 3 8 1 .0 0 0 5 8 4 7 7 .4 3 8 E rro r(T IM E)Sp h e ric ity A ssu m e d 3 8 0 4 3 .5 9 9 297 1 2 8 .0 9 3 G re e n h o u se -G e isse r 3 8 0 4 3 .5 9 9 1 6 8 .2 7 5 2 2 6 .0 8 0 H u yn h -F e ld t 3 8 0 4 3 .5 9 9 1 7 0 .9 0 9 2 2 2 .5 9 6 L o w e r-b o u n d 3 8 0 4 3 .5 9 9 9 9 .0 0 0 3 8 4 .2 7 9 S o u rc e T IM E F 1 5 2 .1 7 5 1 5 2 .1 7 5 1 5 2 .1 7 5 1 5 2 .1 7 5 S ig . Eta S q u a re d .0 0 0 .6 0 6 .0 0 0 .6 0 6 .0 0 0 .6 0 6 .0 0 0 .6 0 6
Tests of Within-Subjects Contrasts M easure: MEASURE_1 Source TIME TIM E Linear Quadratic Cubic Error(TIM E) Linear Quadratic Cubic Type III Sum of Squares 43912.459 12795.944 1769.034 19332.014 13544.132 5167.453 df 1 1 1 99 99 99 M ean Square 43912.459 12795.944 1769.034 195.273 136.809 52.196 F 224.877 93.531 33.892 Sig. .000 .000 .000 Eta Squared .694 .486 .255
Tests of Between-Subjects Effects Measure: MEASURE_1 Transformed Variable: Average Type III Sum Source of Squares Intercept 3018286.453 Error 17285.745 df 1 99 Mean Square F 3018286.453 17286.519 174.603 Sig. .000 Eta Squared .994
8 INTERPRETATION OF OUTPUT FROM ONE-WAY REPEATED MEASURES ANOVA The sphericity assumption requires that the variance of the population difference scores for any two conditions are the same as the variance of the population difference scores for any other two conditions (an assumption that is commonly violated). This is assessed by SPSS using Mauchlys Test of Sphericity. The multivariate statistics however do not require sphericity. In this example, we have violated the assumption of sphericity, as indicated by the Sig. value of 0.000 in the box labeled Mauchlys Test of Sphericity. Although there are ways to compensate for this assumption violation, it is safer to inspect the multivariate statistics provided in the output. Lets look at the key values in the output that you will need to consider. Descriptive statistics In the first output box you are provided with the descriptive statistics for your four sets of scores (mean, standard deviation, N). It is a good idea to check that these make sense. Are there the right number of people in each group? Do the mean values make sense given the scale that was used? In the example above you will see that the lowest mean of VF-14 score was for time 1 (pre-operation) and the highest at time 4 (after six months operation). Multivariate tests In this table, the value that you are interested in is Wilks Lambda and the associated probability value given in the column labeled Sig. All of the multivariate tests yield the same result, however the most commonly reported statistic is Wilks Lambda. In this example the value for Wilks Lambda is 0.302, with a probability value of 0.000 (which really means p<0.0005). The p value is less than 0.05, therefore we can conclude that there is a statistically significant effect for time. This suggests that there was a change in VF-14 scores across the four different time periods. Effect size Although we have found a statistically significant difference between the four sets of scores, we also need to assess the effect size of this result. The value you are interested in is Eta squared, given in the Multivariate Tests output box. The value obtained in this study is 0.698. Using the commonly used guidelines proposed by Cohen (1988), this result suggests a very large effect size (0.01 = small effect, 0.06 = moderate effect, 0.14 = large effect) . If you obtain a statistically significant result from the above analyses, this suggests that there is a difference somewhere among your groups. It does not tell you which groups or set of scores (Time 1, Time 2, Time 3, Time 4) differ from one another. It is necessary to conduct further post-hoc tests to identify which groups differed.
9 PRESENTING THE RESULT FROM ONE-WAY REPEATED MEASURES ANOVA The result of one-way repeated measures ANOVA could be presented as follows: A one-way repeated measures ANOVA was conducted to compare scores on the VF-14 questionnaire Time 1 (prior to the cataract operation), Time 2 (one week post-operation), Time 3 (two months post-operation), and at Time 4 (six months post-operation). The means and standard deviation are presented in Table 1. There was a significant effect for time, Wilks Lambda = 0.302, F = 74.729, p<0.0005, multivariate eta squared = 0.698. Table 1: Descriptive statistics for VF-14 scores for pre-operation, one week, two months and six months post-operation. Time period Time 1 (pre-operation) Time 2 (one-week post-operation) Time 3 (two months post-operation) Time 4 (six months post-operation) N 100 100 100 100 Mean 66.2124 90.6578 94.3863 96.2079 Standard deviation 19.2653 10.1244 6.9793 6.0429
10 TWO-WAY BETWEEN-GROUPS ANOVA Two-way means that there are two independent variables, and between-groups indicates that different people are in each of the groups. This technique allows us to look at the individual and joint effect of two independent variables on one dependent variable. The advantage of using a two-way design is that we can test the main effect for each independent variable and also explore the possibility of an interaction effect. An interaction effect occurs when the effect of one independent variable on the dependent variable depends on the level of a second independent variable. For example, in this case we may find that the influence of age on quality of life is different for males and females. For males, quality of life may increase with age (life begins at 40!), while for females it may decrease. If that was the case, we would say that there is an interaction effect. In order to describe the impact of age, we must specify which group (males/females) we are referring to. Example: A total of 100 patients who needed cataract operation and fulfilled the inclusion and exclusion criteria were selected from patients who were referred to HUKM. These patients were randomly allocated into two groups; 50 in ECCE and 50 in Phaco group. Effectiveness of cataract operation was assessed by generic quality of life questionnaire (Short-from 36, SF-36). It has 8 dimensions of quality of life: emotional role, physical role, pain, vitality, general health profile, social, physical and mental functioning. The score produced by this questionnaire ranges from 0 (unable to do all applicable activities) and a maximum of 100 (able to do all applicable items without difficulty). SF-36 questionnaire was administered prior to operation, one week, two months and six months after operation. For the analysis we will use physical dimension of quality of life since it is relevant with the vision functioning. Details of the variables names and labels from the data file are as follows: File name CIS Variable Variable label name phys1 Quality of life (physical dimension) preoperation. age.grp Age coded into 3 groups Coding instructions Total score on the quality of life (physical dimension) before cataract operation. Scores can range from 0 to 100 with high scores indicating higher levels of quality of life. This variable is a recoded variable, dividing age into three equal groups: Group 1: 45 <60 = 1 Group 2: 60 <75 = 2 Group 3: 75 + = 3 Males = 1 , Females = 2
sex
Sex
11
Research question: What is the impact of age and gender on quality of life (physical dimension)? Does gender moderate the relationship between age and quality of life (physical dimension)? This example involves three variables: a) Two categorical independent variables (e.g: sex: males/females; age group: group 1, 2 & 3) b) One continuous dependent variable (e.g: quality of life (physical dimension) phys1) Two-way ANOVA allows you to simultaneously test for the effect of each of your independent variables on the dependent variable and also identifies any interaction effect. For example, it allows you to test for: a) Sex difference in quality of life (physical dimension) b) Differences in quality of life (physical dimension) for age group 1, 2, and 3 patients c) The interaction of these two variables is there a difference in the effect of age on quality of life (physical dimension) for males and females? Assumptions: a) Normal distribution a. Kolmogorov-Smirnov statistic (a non-significant result, p>0.05 indicates normality) b. Skewness and kurtosis values (if the distribution is perfectly normal, the value of skewness and kurtosis are 0) c. Histograms (appear to be reasonably normally distributed) d. Normal Q-Q Plots (a reasonably straight line between observed value for each score against the expected value from the normal distribution, suggests a normal distribution) e. Detrended Normal Q-Q Plots (there should be no real clustering of points, with most collecting around the zero line) f. Boxplot (the rectangle represents 50% of the cases, with the whiskers (the lines protruding from the box) going out to the smallest and largest values. Additional circles outside the range is called outliers. The line inside the rectangle is the median value.
12 b) Homogeneity of variance The samples are assumed to be obtained from populations of equal variances. This means that the variability of scores for each of the groups is similar. Levenes test can be performed to test the equality of variances as part of the t-test and analysis of variances analyses. A significance value of less than 0.05, suggests that variances for the two groups are not equal, and therefore the assumption of homogeneity of variance is violated. Analysis of variance is reasonably robust to violations of this assumption, provided the size of the groups is reasonably similar (largest/smallest = 1.5).
PROCEDURE FOR TWO-WAY ANOVA 1. Open file CIS 2. From the menu, click: Analyze General Linear Model Univariate 3. Click on your dependent variable (phys1) and move it into the box labeled Dependent variable. 4. Click on your two independent variables (sex, age.grp) and move these into the box labeled Fixed factors. 5. Click on the Options button. - Click on Descriptive Statistics, Estimates of effect size, Observed power, and Homogeneity tests. - Click on Continue 6. Click on the Post Hoc button. - From the Factors listed on the left-hand side choose the independent variable(s) you are interested in (this variable should have three or more levels or groups: eg: agegrp) - Click on the arrow button to move it into the Post Hoc Tests for section - Choose the test you wish to use (eg: Tukey) - Click on Continue 7. Click on the Plots button. - In the Horizontal box put the independent variable that has the most groups (eg: age.grp). - In the box labeled Separate Lines put the other independent variable (eg: sex) - Click on Add. - In the section labeled Plots you should now see your two variables listed (eg: age.grp*sex) 8. Click on Continue and then OK.
Descriptive Statistics Dependent Variable: PHYS1 Sex of patient AGE.GRP Male 1 2 3 Total Female 1 2 3 Total Total 1 2 3 Total Mean 90.5882 80.8000 55.0000 84.0698 85.0000 84.6053 50.8333 81.1404 88.1667 83.0952 51.4286 82.4000 Std. Deviation 21.4973 18.2962 . 20.2736 15.9426 17.1394 20.3511 19.8885 19.1853 17.5616 18.6445 20.0061 N 17 25 1 43 13 38 6 57 30 63 7 100
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+SEX+AGE.GRP+SEX * AGE.GRP
14
T e s t s o f B e tw e e n - S u b je c t s E f fe c ts D e p e n d e n t V a r ia b le : P H Y S 1 T y p e III S u m So urce o f S q u a re s C o r re c te d M o d e l 8 2 0 5 .9 7b0 In te rc e p t 1 4 5 8 6 8 .3 2 6 SEX 2 5 .8 6 2 A G E .G R P 3 7 8 1 .0 9 2 S E X * A G E .G R P 4 5 6 .4 2 0 E rr o r 3 1 4 1 8 .0 3 0 T o ta l 7 1 8 6 0 0 .0 0 0 C o r re c te d T o ta l 3 9 6 2 4 .0 0 0 df 5 1 1 2 2 94 100 99 M e a n Sq u are F 1 6 4 1 .1 9 4 4 .9 1 0 1 4 5 8 6 8 .3 2 6 4 3 6 .4 2 5 2 5 .8 6 2 .0 7 7 1 8 9 0 .5 4 6 5 .6 5 6 2 2 8 .2 1 0 .6 8 3 3 3 4 .2 3 4 N o n c e n t. O b se r v e d a S ig . E ta S q u a r e d P a r a m e te r P o w e r .0 0 0 .2 0 7 2 4 .5 5 2 .9 7 7 .0 0 0 .8 2 3 4 3 6 .4 2 5 1 .0 0 0 .7 8 1 .0 0 1 .0 7 7 .0 5 9 .0 0 5 .1 0 7 1 1 .3 1 3 .8 5 1 .5 0 8 .0 1 4 1 .3 6 6 .1 6 2
a . C o m p u te d u s in g a lp h a = .0 5 b . R S q u a re d = .2 0 7 ( A d ju s te d R S q u a re d = .1 6 5 )
Multiple Comparisons Dependent Variable: PHYS1 Tukey HSD Mean Difference (I-J) Std. Error 5.0714 4.0554 36.7381* 7.6739 -5.0714 4.0554 31.6667* 7.2838 -36.7381* 7.6739 -31.6667* 7.2838
95% Confidence Interval Lower Bound Upper Bound -4.5862 14.7290 18.4634 55.0128 -14.7290 4.5862 14.3211 49.0122 -55.0128 -18.4634 -49.0122 -14.3211
Based on observed means. *. The mean difference is significant at the .05 level.
15
Homogeneous Subsets
PHYS1 Tukey HSD AGE.GRP 3 2 1 Sig.
a,b,c
1.000
Means for groups in homogeneous subsets are displayed. Based on Type III Sum of Squares The error term is Mean Square(Error) = 334.234. a. Uses Harmonic Mean Sample Size = 15.620. b. The group sizes are unequal. The harmonic mean of the group sizes is used. Type I error levels are not guaranteed. c. Alpha = .05.
Profile Plots
90
80
70
60
Sex of patient
50 Male Female 1 2 3
40
AGE.GRP
16 INTERPRETATION OF OUTPUT FROM TWO-WAY ANOVA The output from ANOVA gives you a number of tables and plots with useful information concerning the effect of your two independent variables. Descriptive statistics These provide the mean scores, standard deviations and N for each subgroup. Check that these values are correct. Levenes Test of Equality of Error Variances This test provides a test of one of the assumptions underlying analysis of variance. The value you are most interested in is the Sig. level. You want this to be greater than 0.05 (not significant). A significant result suggest that the variance of your dependent variable across the group is not equal. In the example displayed above the Sig. level is 0.667. As this is larger than 0.05, we can conclude that we have not violated the homogeneity of variance assumption. Main effects The main output from two-way ANOVA is a table labeled Tests Of Between-Subjects Effects. In the left-hand column the independent variables are listed. To determine if there is a main effect for each independent variable, check in the column marked Sig. next to each variable. If the value is less than 0.05, then there is a significant main effect for that independent variable. In the example shown, there is a significant main effect for age group (p=0.005), but no significant main effect for sex (p=0.781). This means that males and females do not differ in terms of their quality of life (physical dimension) scores, however there is a difference in scores for group 1, 2 and 3 patients. Effect size The effect size for the age.grp variable is provided in the column labeled Eta Squared (0.107). Using Cohens criterion, this can be classified as moderate to large effect. This means that the difference in quality of life (physical dimension) between the groups is moderate to large difference and its significant. Interaction effects SPSS tells you whether there is an interaction between the two independent variables in their effect on the dependent variable. The line you need to look at in the table lists your two independent variables separated by an asterisk (sex*age.grp). To find out if the interaction is significant, check the Sig. column for that line. If the value is less than 0.05, then there is a significant interaction effect. In the example, the interaction effect is not significant (sex*age.grp: p=0.508). This indicates that there is no significant
17 difference in the effect of age on quality of life (physical dimension) for males and females. Post-hoc tests Although we know that our age group differ, we do not know where these difference occur: is group 1 different to group 2, is group 2 different to group 3, or is group 1 different to group 3 ? To investigate these questions, we need to conduct post-hoc tests. Post-hoc tests are only relevant if you have more than two level (groups) to your independent variable. However you are not supposed to look at them, until you find a significant main effect or interaction effect in the overall analysis. In this example, we obtained a significant main effect for age.grp in our ANOVA, therefore we are entitled to dig further using the post-hoc tests for age.grp. Multiple comparisons The results of the post-hoc tests are provided in the table labeled Multiple Comparisons. We have requested the Tukey, as this is one of the more commonly used tests. Look down the column labeled Sig. for any values less than 0.05 (indicated by an asterisk). In the example, only group 1 (45 <60) and group 3 (75 +), and group 2 (60 - <75) and group 3 (75 +) differ significantly from one another. Plots You will see at the end of your SPSS output a plot of the optimism scores for males and females, across the three age groups. This plot is very useful for allowing you to visually inspect the relationship among your variables. The plots are often useful to inspect first and understand the impact of your two independent variables. PRESENTING THE RESULT FROM TWO-WAY ANOVA The result of the analysis conducted above could be presented as follows: A two-way between-group analysis of variance was conducted to explore the impact of sex and age on level of quality of life (physical dimension), as measured by the Short-Form 36 questionnaire (SF-36). Subjects were divided into three groups according to their age (group 1: 45 <60 years; group 2: 60 <75 years; group 3: 75 years and above). There was a statistically significant main effect for age (F = 5.656, p = 0.005), with the effect size moderate to large (eta squared = 0.107). Post-hoc comparisons using the Tukey HSD test indicated that the mean score for the 45 <60 years age group (mean = 88.17, s.d = 19.19) was significantly different from the 75 + years group (mean = 51.43, s.d = 18.64), and between the 60 - < 75 years age group (mean = 83.10, s.d = 17.56) and 75 + age group. The main effect for sex (F = 0.077, p = 0.781) and the interaction effect (F = 0.683, p = 0.508) did not reach statistical significance.
18 MIXED BETWEEN-WITHIN SUBJECTS ANALYSIS OF VARIANCE In the previous analysis of variance chapters we have explored the use of both betweensubjects designs (comparing two or more different groups), and within-subjects or repeated measures designs (one group of subjects exposed to two or more conditions). There may be situations however, where you want to combine the two approaches in the one study, with one independent variable being between-subjects, and the other a withinsubjects variable. For example, you may want to investigate the impact of a cataract operation on patients vision quality of life levels (using pre-test and post-test), but you would also like to know if the impact is different for ECCE and Phaco. In this case you have two independent variables, one is a between-subjects variable (type of operation), the other variable is a within-subjects variable (time). In this case you would perform ECCE and Phaco, and measure their quality of life before and after operation) Example: This data (CIS) refers to a true study which involves testing the outcome of two different types of operation (ECCE and Phaco) in treating cataract. Patients were divided into two equal groups and asked to complete vision related quality of life questionnaire (Visual Function-14, VF-14) before operation. After the operation, the patients were again asked to complete the same questionnaire one week after the operation. VF-14 was also administered during their followed-up two and six months later. In this example we will compare the outcome of ECCE and Phaco on patients scores on the VF-14 questionnaire, across the four time periods. Details of the variables names and labels from the data file are provided in the following table. File CIS Variable Variable label type.op Type of operation vf.preop VF-14 scores pre-operation vf.week1 VF-14 scores one week postoperation vf.mnth2 VF-14 scores two months post-operation vf.mnth6 VF-14 scores six months post-operation Coding instructions 1 = ECCE 2 = Phacoemulsification (Phaco) Total scores on the visual functioning administered prior to the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered one week after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered 2 months after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning. Total scores on the visual functioning administered 6 months after the operation. Scores range from 0 100. High scores indicate higher levels of visual functioning.
19
Research question: Which type of cataract operation is more effective in increasing patients vision related quality of life scores across the four time periods (pre-operation, one week, two months and six months post-operation)? Is there a change in patients vision related quality of life scores across the four time periods (before operation, one week, two months and six months after the operation)? This example involves three variables: a) One categorical independent between-subjects variable with two or more levels ECCE / Phaco b) One categorical independent within-subjects variable with two or more levels vf.preop / vf.week1 / vf.mnth2 / vf.mnth6 c) One continuous dependent variable (scores on the VF-14 questionnaire measured at each time period) Mixed between-within ANOVA will test whether there are main effects for each of the independent variables and whether the interaction between the two variables is significant. In this example it will tell us if there is a change in VF-14 scores over the four time periods (main effect for time). It will compare the two operations (ECCE and Phaco) in terms of their effectiveness in increasing the vision related quality of life (main effect for group). Finally it will tell us if the change in VF-14 scores over time is different for the two groups (interaction effect). Assumptions: a) Normal distribution a. Kolmogorov-Smirnov statistic (a non-significant result, p>0.05 indicates normality) b. Skewness and kurtosis values (if the distribution is perfectly normal, the value of skewness and kurtosis are 0) c. Histograms (appear to be reasonably normally distributed) d. Normal Q-Q Plots (a reasonably straight line between observed value for each score against the expected value from the normal distribution, suggests a normal distribution) e. Detrended Normal Q-Q Plots (there should be no real clustering of points, with most collecting around the zero line) f. Boxplot (the rectangle represents 50% of the cases, with the whiskers (the lines protruding from the box) going out to the smallest and largest values. Additional circles outside the range is called outliers. The line inside the rectangle is the median value.
20 b) Homogeneity of variance The samples are assumed to be obtained from populations of equal variances. This means that the variability of scores for each of the groups is similar. Levenes test can be performed to test the equality of variances as part of the t-test and analysis of variances analyses. A significance value of less than 0.05, suggests that variances for the two groups are not equal, and therefore the assumption of homogeneity of variance is violated. Analysis of variance is reasonably robust to violations of this assumption, provided the size of the groups is reasonably similar (largest/smallest = 1.5). c) Homogeneity of intercorrelations For each of the levels of the between-subjects variable the pattern of intercorrelations among the levels of the within-subjects variable should be the same. This assumption is tested as part of the analysis, using Boxs M statistic. Because this statistic is very sensitive, a more conservative alpha level of 0.001 should be used. You hope that the statistic is not significant (p > 0.001).
PROCEDURE FOR MIXED BETWEEN-WITHIN ANOVA 1. Open file CIS 2. From the menu, click: Analyze General Linear Model Repeated Measures 3. In the Within-Subject Factor Name box, type in a name that describes the within-subjects factor (time). This is not an actual variable name, just a label you give your independent variable. 4. In the Number of Levels box type the number of levels or groups (time periods) involved (in this example it is four). 5. Click Add. 6. Click on the Define button on the right-hand side. 7. Click on the variables that represent the within-subjects factor (vf.preop, vf.week1, vf.mnth2, vf.mnth6). 8. Click on the arrow to move these into the Within-Subjects Variables box. You will see them listed. 9. Click on your between-subjects variable (type.op). Click on the arrow to move this variable into the Between-Subjects Factors box. 10. Click on the Options box at the bottom right of your screen. - In the Display section, click on Descriptive statistics, Estimates of effect size, Observed power and Homogeneity tests. - Click on Continue.
21 11. Click on the Plots button. - Click on the within-groups factor (time) and move it into the box labeled Horizontal Axis. - In the Separate Lines box click on the between-groups variable (type.op) 12. Click on Add. In the box, you should see your variables listed (time*type.op). 13. Click on Continue and then OK. The output generated from this procedure is shown below. Only a selected sample of the output is displayed.
Descriptive Statistics Type of Operation ECCE Phaco Total VF-14 score - week 1 ECCE Phaco Total VF-14 score - month 2 ECCE Phaco Total VF-14 score - month 6 ECCE Phaco Total VF-14 score - pre operation Mean 64.0544 68.3704 66.2124 89.2129 92.1026 90.6578 93.4333 95.3394 94.3863 96.9506 95.4651 96.2079 Std. Deviation 19.7875 18.6770 19.2653 11.4170 8.5140 10.1244 6.9733 6.9234 6.9793 5.8804 6.1704 6.0429 N 50 50 100 50 50 100 50 50 100 50 50 100
22
a Box's Test of Equality of Covariance Matrices
Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups. a. Design: Intercept+TYPE.OP Within Subjects Design: TIME M u lt iv a r ia t e c e s t s T E ffe c t T IM E V a lu e P illa i's T r a c e .7 0 4 W ilk s ' L a m b d a .2 9 6 H o te llin g 's T r a c e 2 .3 7 3 R o y 's L a r g e s t R o o2t .3 7 3 T IM E * T Y P E .Oilla i's T r a c e P P .0 6 4 W ilk s ' L a m b d a .9 3 6 H o te llin g 's T r a c e .0 6 8 R o y 's L a r g e s t R o o t.0 6 8 a . C o m p u te d u s in g a lp h a = .0 5 b . E x a c t s ta tis tic c. D e s ig n : In te r c e p t+ T Y P E .O P W ith in S u b je c ts D e s ig n : T IM E
M a u c h ly 's T e s t o f Sp h ebric it y M e a su re : M EASU R E_ 1 Ep silo n Ap p ro x. W ith in Su b je cts EffeM a u ch ly's W C h i-Sq u a re ct T IM E .2 7 1 1 2 6 .2 5 6 df 5 S ig. .0 0 0 G re e n h o u s e -G e isse r H u yn h - F e ld tL o w e r-b o u n d .5 6 2 .5 7 6 .3 3 3
a
N o n c e n t. O b s e r v e d a S ig . E ta S q u a r e P a r a m e te r P o w e r d .0 0 0 .7 0 4 2 2 7 .8 2 4 1 .0 0 0 .0 0 0 .7 0 4 2 2 7 .8 2 4 1 .0 0 0 .0 0 0 .7 0 4 2 2 7 .8 2 4 1 .0 0 0 .0 0 0 .7 0 4 2 2 7 .8 2 4 1 .0 0 0 .0 9 5 .0 6 4 6 .5 4 4 .5 3 9 .0 9 5 .0 6 4 6 .5 4 4 .5 3 9 .0 9 5 .0 6 4 6 .5 4 4 .5 3 9 .0 9 5 .0 6 4 6 .5 4 4 .5 3 9
T e sts th e n u ll h yp o th e sis th a t th e e rro r co va ria n ce m a trix o f th e o rth o n o rm a lize d tra n sfo rm e d d e p e n d e n t va ria b le s is p ro p o rtio n a l to a n id e n tity m a trix. a . M ay b e u se d to a d ju st th e d e g re e s o f fre e d o m fo r th e a ve ra g e d te sts o f sig n ifica n ce . C o rre cte d te sts a re d isp la ye T e sts o f W ith in -Su b je cts Effe cts ta b le . b. D e sig n : In te rce p t+T YPE.O P W ith in Su b je cts D e sig n : T IM E
23
24
a Levene's Test of Equality of Error Variances
F VF-14 score operation VF-14 score VF-14 score VF-14 score - pre - week 1 - month 2 - month 6 .029 1.017 .451 1.273
df1 1 1 1 1
df2 98 98 98 98
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+TYPE.OP Within Subjects Design: TIME T e s ts o f B e tw e e n -Su b je c ts Effe c ts M e a su re : M EASU R E_ 1 T ra n sfo rm e d Va ria b le : Ave ra g e T yp e III Su m So u rce o f Sq u a re s In te rce p t 3 0 1 8 2 8 6 .4 5 3 T YP E.O P 3 6 3 .5 0 4 Erro r 1 6 9 2 2 .2 4 1 df M e a n Sq u a re F 1 3 0 1 8 2 8 6 .4 5 31 7 4 7 9 .4 8 6 1 3 6 3 .5 0 4 2 .1 0 5 98 1 7 2 .6 7 6 N o n ce n t. O b se rve d a Sig . Eta Sq u a re d Pa ra m e te r Po w e r .0 0 0 .9 9 4 1 7 4 7 9 .4 8 6 1 .0 0 0 .1 5 0 .0 2 1 2 .1 0 5 .3 0 1
a . C om p u te d u sin g a lp h a = .0 5
Profile Plots
90
80
70
Type of Operation
ECCE
60 1 2 3 4
Phaco
TIME
25
INTERPRETATION OF OUTPUT FROM MIXED BETWEEN-WITHIN ANOVA This output provides tests for the assumptions of sphericity, univariate ANOVA results and also multivariate ANOVA results. The sphericity assumption requires that the variance of the population difference scores for any two conditions are the same as the variance of the population difference scores for any other two conditions (an assumption that is commonly violated). This is assessed by SPSS using Mauchlys Test of Sphericity. The multivariate statistics however do not require sphericity. In this example, we have violated the assumption of sphericity, as indicated by the Sig. value of 0.000 in the box labeled Mauchlys Test of Sphericity. Although there are ways to compensate for this assumption violation, it is safer to inspect the multivariate statistics provided in the output. Lets look at the key values in the output that you will need to consider. Descriptive statistics In the first output box you are provided with the descriptive statistics for your four sets of scores (mean, standard deviation, N). It is a good idea to check that these make sense. Are there the right number of people in each group? Do the mean values make sense given the scale that was used? In the example above you will see that the lowest VF-14 scores are at pre-operation (ECCE = 64.05 and Phaco = 68.37), they increased at one week post-operation (89.21 and 92.10) and increased even further at two months (93.43 and 95.34) and six months (96.95 and 95.47) after operation. What we dont know however, is whether these differences are large enough to be considered statistically significant. Multivariate tests In this table, the value that you are interested in is Wilks Lambda and the associated probability value given in the column labeled Sig. All of the multivariate tests yield the same result, however the most commonly reported statistic is Wilks Lambda. In this example the value for Wilks Lambda is 0.296, with a probability value of 0.000 (which really means p<0.0005). The p value is less than 0.05, therefore we can conclude that there is a statistically significant effect for time. This suggests that there was a change in VF-14 scores across the four different time periods. The main effect for time was significant. Effect size Although we have found a statistically significant difference between the four sets of scores, we also need to assess the effect size of this result. The value you are interested in is Eta squared, given in the Multivariate Tests output box. The value obtained in
26 this study is 0.704. Using the commonly used guidelines, this result suggests a very large effect size (0.01 = small effect, 0.06 = moderate effect, 0.14 = large effect) . This result suggests a very large effect size. Interaction effect In addition to the main effect, we are also interested to find out if there is a significant interaction effect. Is there the same change in scores over time for the two different types of operations (ECCE / Phaco)? This is indicated in the second set of row in the Multivariate Tests table (TIME*TYPE.OP). In this case the interaction effect is not statistically significant (sig. level for Wilks Lambda is 0.095 which is greater than our alpha level of 0.05) We have explored the within-subjects effects, now we need to consider the main effect of our between-subjects variable (type of operation: ECCE / Phaco). Between-subjects effect The results that we need to look at are in the table labeled Tests of Between-Subjects Effects. Read across the row labeled TYPE.OP (shortened for the type of operation). The Sig. value is 0.150. This is not less than our alpha level of 0.05, therefore we conclude that the main effect for group is not significant. There was no significant difference in the VF-14 scores for the two types of operation. Effect size The effect size of the between-subject effect is also given in the Tests Of BetweenSubject Effects table. The eta-squared value for group in this case in 0.021. This is small and therefore not surprising that it did not reach statistical significance.
PRESENTING THE RESULT FROM MIXED BETWEEN-WITHIN ANOVA The method of presenting the results from this technique is a combination of that used for a between-groups ANOVA, and a repeated-measures ANOVA. Report the main effects for each independent variable and associated effect sizes, and the interaction effect.
27
MULTIVARIATE ANALYSIS OF VARIANCE In previous chapters we explored the use of analysis of variance to compare groups on a single dependent variable. In many research situations however we are interested in comparing groups on a range of different characteristics. This is quite common in clinical research where the focus is on the evaluation of the impact of an intervention on a variety of outcome measures (eg. anxiety, depression, physical symptoms, diagnosis and quality of life). Multivariate analysis of variance (MANOVA) is an extension of analysis of variance for use when you have more than one dependent variable. These dependent variables should be related in some way, or there should be some conceptual reason for considering them together. MANOVA compares the groups and tells you whether the mean differences between the groups on the combination of dependent variables is likely to have occurred by chance. Some of you might be thinking why not just conduct a series of ANOVAs separately for each dependent variable. This is in fact what many researchers do. Unfortunately by conducting a whole series of analyses you run the risk of an inflated Type 1 error. This means that the more analyses you run, the more likely you are to find a significant result, even if in reality, there are no real differences between your groups. However it has a number of additional assumptions that must be met. MANOVA can be used in one-way, two-way and higher factorial designs (with multiple independent variables), and when using analysis of covariance (controlling for an additional variable). SUMMARY FOR ONE-WAY MANOVA Research question: Do patients in ECCE and Phaco groups differ in terms of overall wellbeing? Are patients in ECCE better in their wellbeing than Phaco in terms of their VF-14, quality of life (physical dimension) and (mental dimension) before undergo cataract operation ? This example involves: a) One categorical independent variable (eg. type of operation) b) Two or more continuous dependent variables (eg. VF-14 (pre-op), quality of life (physical dimension) and (mental dimension) before operation). MANOVA can also be extended to two-way and higher order designs involving two or more categorical independent variables. MANOVA compares two or more groups in terms of their means on a group of dependent variables
28 Assumptions: Before proceeding with the main MANOVA analysis we will test whether our data conforms to the assumptions required. Some of these tests are not strictly necessary provided that our sample size is large enough. a) Sample size Having a large sample can also help you get away with violations of some of the other assumptions (eg. normality). The minimum required number of cases in each cell in this example is three (the number of dependent variables). We have a total of six cells (two levels of independent variable (ECCE/Phaco); and three dependent variables for each). b) Normality Although the significance tests of MANOVA are based on the multivariate normal distribution, in practice it is reasonably robust to modest violations of normality (except where the violations are due to outliers). According to Tabachnick and Fidell (1996), a sample size of at least 20 in each cell should ensure robustness. You need to check both univariate normality and multivariate normality (using Mahalanobis distances). c) Outliers MANOVA is quite sensitive to outliers (data points or scores that are different from the remainder of the scores). Check for outliers by using Explore. Procedures to identify outliers are as follow: 1. From the menu at the top of the screen, click on: Analyze Descriptive Statistics Explore 2. Click on the variable that you want to detect any outliers. Move this into the Dependents box. 3. Click on your identifying variable (eg. ID, no.resp), and move it into the box labeled Label Cases by. 4. Click on the Statistics button. Tick Descriptive and Outliers. 5. Click on Continue and then OK. The Extreme Values box gives you the highest and lowest values for the variable and the ID number of the sample that recorded these scores. If the score is not too high/low, we will use the data. If there had been a lot of outlying cases, we may need to consider transforming this group of variables or remove the cases from the data file.
29 d) Linearity This assumption refers to the presence of a straight-line relationship between each pair of your dependent variables. This can be assessed in a number of ways, the most straightforward of which is to generate scatterplots between each pair of your variables. This involves splitting the file by type of operation, and then generating scatterplots. Procedure to split the file: 1. From the menu at the top of the screen click on: Data, then click on Split File. 2. Click on Organize output by groups. 3. Click on your independent categorical variable that will be used to create the group (eg. type.op). 4. Move this variable into the box labeled; Groups based on. Click on OK. Procedure to generate scatterplots: Having split your file, you can now request your scatterplots. You will need to repeat this step to cover all possible pairs of your dependent variables. 1. 2. 3. 4. From the menu at the top of the screen click on: Graphs, then click on Scatter. Click on Simple. Click on the Define button. Click on one of your dependent variables and move it into the box labeled Y axis. Click on another of your dependent variables and move it into the box labeled X axis (it does not matter in which order you enter your variables). Click on OK. 5. Remember to go back and turn your Split File option off when you have finished. Click on Data, then click on Split File. 6. Click on Analyze all cases, do not create groups. Click on OK.
30
Graph
TYPE.OP:
120
1 ECCE
100
80
60
40
20
0 0 20 40 60 80 100 120
PHYS1
TYPE.OP:
120
2 Phaco
100
80
60
40
20 20 40 60 80 100 120
PHYS1
These plots do not show any evidence of non-linearity, therefore our assumption of linearity is satisfied.
31
e) Homogeneity of regression This assumption is important if you are intending to perform a stepdown analysis. This approach is used when you have some theoretical or conceptual reason for ordering your dependent variables and this is quite a complex precedure. f) Multicollinearity and singularity MANOVA works best when the dependent variables are only moderately correlated. With low correlations you should consider running separate univariate analysis of variance for your various dependent variables. When the dependent variables are highly correlated this is referred to as multicollinearity. This can occur when one of your variables is a combination of other variables (eg. the total scores of a scale that is made up of subscales that are also included as dependent variables). This is referred to as singularity, and can be avoided by knowing what your variables are, and how the scores are obtained. While there are quite sophisticated ways of checking for multicollinearity, the simplest way is to run Correlation and to check the strength of the correlations among your dependent variables. Correlations up around 0.8 or 0.9 are reason for concern. If you any of these, you may need to consider removing one of the strongly correlated pairs of dependent variables, or alternatively combining them to form a single measure. g) Homogeneity of variance-covariance matrices Fortunately, the test of this assumption is generated as part of your MANOVA output. The test used to assess this is Boxs M Test of Equality of Covariance Matrices. PROCEDURE FOR MANOVA 1. Open file CIS 2. From the menu, click: Analyze General Linear Model Multivariate 3. In the Dependent Variables box enter each of your dependent variables (eg. VF14 pre-op, Quality of life (physical dimension) and Quality of life (mental dimension) before operation). 4. In the Fixed Factors box enter your independent variable (eg. type.op). 5. Click on the Model button. Make sure that the Full Factorial button is selected in the Specify Model box. 6. Down the bottom in the Sum of Squares box, Type III should be displayed. This is the default method of calculating sums of squares. Click on Continue.
32 7. Click on the Options button. In the section labeled Factor and factor Interactions click on your independent variable (eg. type.op). Move it into the box marked Display Means for: 8. In the Display section of this screen, put a tick in the boxes labeled: a. Descriptive Statistics b. Estimated of Effect Size c. Observed Power d. Homogeneity tests 9. Click on Continue and then OK. The output generated from this procedure is shown below.
Descriptive Statistics VF-14 score pre operation PHYS1 Type of Operation ECCE Phaco Total ECCE Phaco Total ECCE Phaco Total Mean 64.0544 68.3704 66.2124 82.4000 82.4000 82.4000 69.3600 66.4000 67.8800 Std. Deviation 19.7875 18.6770 19.2653 20.3098 19.9039 20.0061 3.9267 5.0467 4.7382 N 50 50 100 50 50 100 50 50 100
MENTAL1
Tests the null hypothesis that the observed covariance matrices of the dependent variables are equal across groups. a. Design: Intercept+TYPE.OP
33
M u lt iv a ria t e Tce s t s E ffe c t V a lu e F H y p o th e s is d fE r r o r d f b In te r c e p t P illa i's T r a c e .9 9 6 8 6 5 7 .9 9 1 3 .0 0 0 9 6 .0 0 0 b W ilk s ' L a m b d a .0 0 4 8 6 5 7 .9 9 1 3 .0 0 0 9 6 .0 0 0 b H o te llin g 's T r a c e 2 7 0 .5 6 2 8 6 5 7 .9 9 1 3 .0 0 0 9 6 .0 0 0 b R o y 's L a r g e s t R o 2 7 0 .5 6 2 8 6 5 7 .9 9 1 ot 3 .0 0 0 9 6 .0 0 0 T Y P E .O PP illa i's T r a c e .1 0 2 3 .6 4 b 1 3 .0 0 0 9 6 .0 0 0 b W ilk s ' L a m b d a .8 9 8 3 .6 4 1 3 .0 0 0 9 6 .0 0 0 H o te llin g 's T r a c e .1 1 4 3 .6 4 b 1 3 .0 0 0 9 6 .0 0 0 b R o y 's L a r g e s t R o o t .1 1 4 3 .6 4 1 3 .0 0 0 9 6 .0 0 0 a . C o m p u te d u s in g a lp h a = .0 5 b . E x a c t s ta tis tic c . D e s ig n : In te r c e p t+ T Y P E .O P
a Levene's Test of Equality of Error Variances
N o n c e n t. O b s e r v e d a S ig . E ta S q u a r e dP a r a m e te r P o w e r .0 0 0 .9 9 6 2 5 9 7 3 .9 7 3 1 .0 0 0 .0 0 0 .9 9 6 2 5 9 7 3 .9 7 3 1 .0 0 0 .0 0 0 .9 9 6 2 5 9 7 3 .9 7 3 1 .0 0 0 .0 0 0 .9 9 6 2 5 9 7 3 .9 7 3 1 .0 0 0 .0 1 5 .1 0 2 1 0 .9 2 4 .7 8 3 .0 1 5 .1 0 2 1 0 .9 2 4 .7 8 3 .0 1 5 .1 0 2 1 0 .9 2 4 .7 8 3 .0 1 5 .1 0 2 1 0 .9 2 4 .7 8 3
df1 1 1 1
df2 98 98 98
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+TYPE.OP
34
T e s t s o f B e t w e e n - S u b je c t s E f f e c t s T y p e II I S u m S o urce D e p e n d e n t V a r ia b le q u a r e s d f of S b C o r r e c t e d M oVdFe-l1 4 s c o r e - p r e 4 6 5 .6 8 3 1 o p e r a t io n PHYS1 .0 0 c 0 1 d0 M ENTAL1 2 1 9 .0 4 1 In t e r c e p t V F -1 4 s co re - p re 4 3 8 4 0 8 .1 0 5 1 o p e r a t io n PHYS1 6 7 8 9 7 6 .0 0 0 1 M ENTAL1 4 6 0 7 6 9 .4 4 0 1 T Y P E .O P V F -1 4 s co re - p re 4 6 5 .6 8 3 1 o p e r a t io n PHYS1 .0 0 0 1 M ENTAL1 2 1 9 .0 4 0 1 E rro r V F -1 4 s co re - p re 3 6 2 7 8 .4 6 8 98 o p e r a t io n PHYS1 3 9 6 2 4 .0 0 0 98 M ENTAL1 2 0 0 3 .5 2 0 98 T o ta l V F -1 4 s co re - p re 4 7 5 1 5 2 .2 5 6 1 0 0 o p e r a t io n PHYS1 7 1 8 6 0 0 .0 0 0 1 0 0 M ENTAL1 4 6 2 9 9 2 .0 0 0 1 0 0 C o r r e c t e d T o VaFl - 1 4 s c o r e - p r e t 3 6 7 4 4 .1 5 1 99 o p e r a t io n PHYS1 3 9 6 2 4 .0 0 0 99 M ENTAL1 2 2 2 2 .5 6 0 99 M e a n Sq u a re F 4 6 5 .6 8 3 1 .2 5 8 S ig . .2 6 5 1 .0 0 0 .0 0 1 .0 0 0 .0 0 0 .0 0 0 .2 6 5 1 .0 0 0 .0 0 1 N o n c e n t. O b s e r v e d a E t a S q u a r eP a r a m e te r P o w e r d .0 1 3 .0 0 0 .0 9 9 1 .2 5 8 .0 0 0 1 0 .7 1 4 .1 9 9 .0 5 0 .9 0 0 1 .0 0 0 1 .0 0 0 1 .0 0 0 .1 9 9 .0 5 0 .9 0 0
.0 0 0 .0 0 0 2 1 9 .0 4 0 1 0 .7 1 4 4 3 8 4 0 8 .1 0 1 1 8 4 . 2 8 4 5 6 7 8 9 7 6 .0 0 1 6 7 9 . 2 7 6 0 4 6 0 7 6 9 .4 4 0 5 3 8 . 0 3 6 22 4 6 5 .6 8 3 1 .2 5 8
.9 2 4 1 1 8 4 .2 8 4 .9 4 5 1 6 7 9 .2 7 6 .9 9 6 2 2 5 3 8 .0 3 6 .0 1 3 .0 0 0 .0 9 9 1 .2 5 8 .0 0 0 1 0 .7 1 4
.0 0 0 .0 0 0 2 1 9 .0 4 0 1 0 .7 1 4 3 7 0 .1 8 8 4 0 4 .3 2 7 2 0 .4 4 4
a .C o m p u t e d u s in g a lp h a = .0 5 b .R S q u a r e d = .0 1 3 ( A d ju s te d R S q u a r e d = .0 0 3 ) c . R S q u a r e d = .0 0 0 ( A d ju s te d R S q u a r e d = - .0 1 0 ) d .R S q u a r e d = .0 9 9 ( A d ju s te d R S q u a r e d = .0 8 9 )
Dependent Variable Type of Operation VF-14 score - pre ECCE operation Phaco PHYS1 ECCE Phaco MENTAL1 ECCE Phaco
35 INTERPRETATION OF OUTPUT FROM MANOVA Descriptive Statistics Check that the information is correct. In particular, check that the N values correspond to what you know about your sample. These N values are your cell size (assumption a). Make sure that you have more subjects (cases) in each cell, than the number of dependent variables. If you have over 30, then any violations of normality or equality of variance that may exist are not going to matter too much. Box Tests The output box labeled Box s Test of Equality of Covariance Matrices will tell you if your data violates the assumption of homogeneity of variance-covariance matrices. If the Sig. value is larger than 0.001, then you have not violated the assumption. Boxs M can tend to be too strict when your have a large sample size. Levenes Test The next box to look at is Levenes Test of Equality of Error Variances. In the Sig. column look for any values that are less than 0.05. These would indicate that you have violated the assumption of equality of variance for that variable. If you do violate this assumption of equality of variances you will need to set a more conservative alpha level -for determining significance for that variable in the univariate F-test (example an alpha of 0.025 or 0.01, rather than the normal conventional 0.05 level). Multivariate tests This set of multivariate tests of significance will indicate whether there are statistically significant differences among the groups on a linear combination of the dependent variables. There are a number of statistics to choose from (Wilks Lambda, Hotellings Trace, Pillais Trace). One of the most commonly reported statistics is Wilks Lambda. It is recommend for general use, however if you data has problems (small sample size, unequal N values, violation of assumptions), then Pillais Trace is more robust. Wilks Lambda You will find the value you are looking for in the second section of the Multivariate Tests table in the row labeled with the name of your independent or grouping variable (in this case: TYPE.OP). Dont make the mistake of using the first set of figures which refer to the intercept. Find the value of Wilks Lambda and its associated significance level (Sig.). If the significance level is less than 0.05, then you can conclude that there is a difference among your groups. In the example shown above, we obtained a Wilks Lambda value of 0.898, with a significance value of 0.015. This is less than 0.05, therefore there is a statistically significant difference between ECCE and Phaco patients in terms of their overall wellbeing.
36
Between-subjects effects If you obtain a significant result on this multivariate test of significance, this then gives you permission to investigate further in relation to each of your dependent variables. Do ECCE and Phaco patients differ on all of the dependent measures, or just some? This information is provided in the Tests of Between-Subjects Effects output box. Because you are looking at a number of separate analyses here, it is suggested that you set a higher alpha level to reduce the chance of a Type 1 error (ie. finding a significant result when in fact there isnt really one). The most common way of doing this is to apply what is known as a Bonferroni adjustment. In its simplest form this involves dividing your original alpha level of 0.05 by the number of analyses that you intend to do. In this case we have three dependent variables to investigate, therefore we would divide 0.05 by 3, giving a new alpha level of 0.017. We will consider our results significant if the probability value (Sig.) is less than 0.017. Significance In the Tests of Between-Subjects Effects box move down to the third set of values in a row labeled with your independent variable (in this case: TYPE.OP). You will see each of your dependent variables listed, with their associated univariate F, df and Sig. values. In the Sig. column look for any values that are less than 0.017 (our new adjusted alpha level). In our example, only one of the dependent variables ie. Quality of life (mental dimension) before operation (MENTAL1) recorded a significance value less than our cutoff (with a Sig. value of 0.001). In this study, the only significant difference between ECCE and Phaco patients was on their Quality of life (mental dimension) before operation. Effect size The importance of the impact of type of operation on Quality if life (mental dimension) can be evaluated using the effect size statistic provided by SPSS: Eta Squared. Although labeled eta squared in the output, the information provided by SPSS on this statistic suggests that it is actually partial eta squared that is given. Both eta squared and partial eta squared represent the proportion of the variance in the dependent variable (Quality of life mental dimension) that can be explained by the independent variable (type of operation). The value in this case is 0.099 which, according to generally accepted criteria, is considered moderate to large effect. This represents that 9.9 percent of the variance in Quality of life (mental dimension) scores explained by type of patient (whether ECCE or Phaco).
37
Comparing group means Although we know that ECCE and Phaco patients differed in terms of Quality of life (mental dimension), we do not know who had the higher scores. To find this out we refer to the output table provided in the section labeled Estimated Marginal Means. For Quality of life (mental dimension) the mean score for ECCE patients was 69.36 and for Phaco patients, 66.40. Although statistically significant, the actual difference in the two mean scores was very small, less than 3 scale points.
PRESENTING THE RESULTS FROM MANOVA The results of this multivariate analysis of variance could be presented as follow: A one-way between-groups multivariate analysis of variance was performed to investigate type of patient (who underwent cataract operation) differences in overall wellbeing. Three dependent variables were used: VF-14 pre-operation, Quality of life (physical dimension) and Quality of life (mental dimension) before operation. The independent variable was type of patient (ECCE / Phaco). Preliminary assumption testing was conducted to check for normality, linearity, univariate and multivariate outliers, homogeneity of variance-covariance matrices, and multicollinearity, with no serious violations noted. There was a statistically significant difference between ECCE and Phaco patients on the combined dependent variables: F = 3.641, p = 0.015; Wilks Lambda = 0.898; partial eta squared = 0.102. When the results for the dependent variables were considered separately, the only difference to reach statistical significance using a Bonferroni adjusted alpha level of 0.017, was Quality of life (mental dimension) before operation: F = 10.714, p = 0.001, partial eta squared = 0.099. An inspection of the mean scores indicated that ECCE patients reported slightly higher levels of Quality of life (mental dimesion) (M = 69.36, s.d = 0.639) than Phaco patients (M = 66.40, s.d = 0.639) before operation.
38 ANALYSIS OF COVARIANCE (ANCOVA) Analysis of covariance is an extension of analysis of variance that allows you to explore differences between groups while statistically controlling for an additional (continuous) variable. This additional variable (called a covariate) is a variable that you suspect may be influencing scores on the dependent variable. SPSS uses regression procedures to remove the variation in the dependent variable that is due to the covariate(s), and then performs the normal analysis of variance techniques on the corrected or adjusted scores. By removing the influence of these additional variables ANCOVA can increase the power or sensitivity of the F-test. This is, it may increase the likelihood that you will be able to detect differences between your groups. Analysis of covariance can be used as part of one-way, two-way and multivariate ANOVA techniques. A) One-way between-groups ANOVA (1 independent variable, 1 dependent variable) B) Two-way between-groups ANOVA (2 independent variables, 1 dependent variable) Uses of ANCOVA ANCOVA can be used when you have a two-group pre-test/post-test design (eg. comparing the impact of two different interventions, taking before and after measures for each group). The scores on the pre-test are treated as a covariate to control for preexisting differences between the groups. This makes ANCOVA very useful in situations when you have quite small sample sizes, and only small or medium effect sizes. Under these circumstances (which are very common in social science research), it is recommends the use of two or three carefully chosen covariates to reduce the error variance and increase your chances of detecting a significant difference between your groups. ANCOVA is also handy when you have been unable to randomly assign your subjects to the different groups, but instead have had to use existing groups (eg. groups of patients). As these groups may differ on a number of different attributes (not just the one you are interested in), ANCOVA can be used in an attempt to reduce some of these differences. The use of well chosen covariates can help to reduce the confounding influence of group differences. This is certainly not an ideal situation, as it is not possible to control for all possible differences, however it does help to reduce this systematic bias. Choosing appropriate covariates ANCOVA can be used to control for one or more covariates at the same time. In identifying possible covariates you should ensure you have a good understanding of the
39 theory and previous research that has been conducted in your topic area. The variables that you choose as your covariates should be continuous variables, measured reliably, and correlate significantly with the dependent variable. Ideally you should choose a small set of covariates that are only moderately correlated with one another, so that each contributes uniquely to the variance explained. The covariate must be measured before the treatment or experimental manipulation is performed. This is to prevent scores on the covariate also being influenced by the treatment. Assumptions of ANCOVA: There are a number of issues and assumptions associated with ANCOVA. These are over and above the usual ANOVA assumptions. a) Influence of treatment on covariate measurement In designing your study you should ensure that the covariate is measured prior to the treatment or experimental manipulation. This is to avoid scores on the covariate also being influenced by the treatment. If the covariate is affected by the treatment condition, then this change will be correlated with the change that occurs in your dependent variable. When ANCOVA removes (controls for) the covariate it will also remove some of the treatment effect, thereby reducing the likelihood of obtaining a significant result. b) Reliability of covariates ANCOVA assumes that covariates are measured without error, which is rather unrealistic assumption in much social science research. Some variables that you may wish to control, such as age, can be measured reasonably reliably; others which rely on a scale, may not meet this assumption. There are a number of things you can do to improve the reliability of your measurement tools: 1. Look for good, well validated scales and questionnaires. Make sure they measure what you think they measure (dont just rely on the title check the manual and inspect the items). 2. Check that the measure you intend to use is suitable for use with your sample (some scales may be very reliable for adults, but may not be suitable for children or adolescents). 3. Check the internal consistency (a form of reliability) of your scale by calculating Cronbach alpha. Value should be above 0.7 or 0.8 to be considered reliable. Be careful with very short scales (eg. under ten items) however, because Cronbach alpha is quite sensitive to the number of items in the scale. Short scales often have low Cronbach alpha values. In this case you may need to check the inter-correlations among the scale items.
40 4. If you have had to write the questions yourself, make sure they are clear, appropriate and unambiguous. Make sure the questions and response scale are appropriate for all of your groups (eg. consider sex differences, language difficulties, cultural backgrounds, etc.). Always pretest your questions before conducting the full study. 5. Consider the circumstances under which you are measuring your covariate. Are people likely to answer honestly, or are they likely to distort their answers to avoid embarrassment etc.? 6. If you are using any form of equipment or measuring instrumentation, make sure that it is functioning properly and calibrated appropriately. Make sure the person operating the equipment is competent and trained in its use. 7. If your study involves using other people to observe or rate behaviour, make sure they are trained and that each observer uses the same criteria. Preliminary pilot testing to check inter-rater consistency would be useful here. c) Correlations among covariates There should not be strong correlations among the variables you choose for your covariates. Ideally, you want a group of covariates that correlate substantially with the dependent variable, but not with one another. To choose appropriate covariates you will need to use the existing theory and research to guide you. Run some preliminary correlation analyses to explore the strength of the relationship among your proposed covariates. If you find that the covariates you intend to use correlate strongly (eg. r = 0.8), you should consider removing one or more of them. Each of the covariates you choose should pull their own weight overlapping covariates do not contribute to a reduction in error variance. d) Linear relationship between dependent variable and covariate ANCOVA assumes that the relationship between the dependent variable and each of your covariates is linear (straight-line). If you are using more than one covariate, it also assumes a linear relationship between each of the pairs of your covariates. Violations of this assumption are likely to reduce the power (sensitivity) of your test. Remember, one of the reasons for including covariates was to increase the power of your analysis of variance test. Scatterplots can be used to test for linearity, however these need to be checked separately for each of your groups. (ie. the different levels of your independent variable). If you discover any curvilinear relationships, these may be corrected by transforming your variable or alternatively you may wish to drop the offending covariate from the analysis. Disposing of covariates that misbehave is often easier, given the difficulty in interpreting transformed variables.
41
e) Homogeneity of regression slopes This impressive sounding assumption requires that the relationship between the covariate and dependent variable for each of your groups is the same. This is indicated by similar slopes on the regression line for each group. Unequal slopes would indicate that there is an interaction between the covariate and the treatment. If there is an interaction, then the results of ANCOVA are misleading, and therefore it should not be conducted. One-way ANCOVA One-way ANCOVA involves one independent, categorical variable (with two or more levels of conditions), one dependent continuous variable, and one or more continuous covariates. This technique is often used when evaluating the impact of an intervention or experimental manipulation, while controlling for pre-test scores. Details of example To illustrate the use of one-way ANCOVA, we will use the CIS data file. This data refers to a study that involves testing the impact of two different types of operation in treating cataract patients. Patients were divided into two equal groups (ECCE and Phaco) and asked to complete a generic quality of life questionnaire (SF-36) before operation. After the operation they were again asked to complete the same questionnaire. In this example we will explore the outcome of ECCE (Group 1) and Phaco (Group 2), on patients scores on the SF-36 questionnaire after the operation, while controlling for the scores on this test administered before the operation. Details of the variables names and labels from the data file are as follows: File name CIS Variable Variable label name type.op Type of operation phys1 SF-36 (physical dimension) scores preoperation phys4 SF-36 (physical dimension) scores 6 months post-operation Coding instructions 1 = ECCE 2 = Phacoemulsification (Phaco) Total scores on the quality of life (physical dimension) administered prior to the operation. Scores range from 0 100. High scores indicate higher levels of quality of life. Total scores on the quality of life (physical dimension) administered 6 months after the operation. Scores range from 0 100. High scores indicate higher levels of quality of life.
42
Summary for one-way ANCOVA Research question: Is there a significant difference in the SF-36 (physical dimension) scores for the ECCE patients (Group 1) and the Phaco patients (Group 2), while controlling for their pre-test scores on this test? At least three variables are involved: 1. One categorical independent variable with two or more levels (ECCE / Phaco) 2. One continuous dependent variable (SF-36 (physical dimension) score six months post-operation) 3. One or more continuous covariates (SF-36 (physical dimension) score preoperation) ANCOVA will tell us if the mean SF-36 (physical dimension) scores at six months postoperation for the two groups (ECCE / Phaco) are significantly different, after the initial pre-test scores are controlled for. Testing assumptions: Before you can begin testing the specific assumptions associated with ANCOVA, you will need to check the assumptions for a normal one-way analysis of variance (normality and homogeneity of variance). Assumption 1: Measurement of the covariate This assumption specifies that the covariate should be measured before the treatment or experimental manipulation begins. This is not tested statistically, but instead forms part of your research design and procedures. This is why it is important to plan your study with a good understanding of the statistical techniques that you intend to use. Assumption 2: Reliability of the covariate This assumption concerning the reliability of the covariate is also part of your research design, and involves choosing the most reliable measuring tools available. If you have used a psychometric scale or measure you can check the internal consistency reliability by calculating Cronbach alpha using the SPSS Reliability procedures.
43 Assumption 3: Correlations amongst the covariates If you are using more than one covariate, you should check to see that they are not too strongly correlated with one another (r = 0.8 and above). To do this you will need to use the SPSS Correlation procedure. Since in this example has one covariate, we do not need to perform the correlation. Assumption 4: Linearity There are a number of different ways that you can check the assumption of a linear relationship between the dependent variable and the covariates for all your groups. One way would be to; 1. Split your sample according to the various groups in your design using the SPSS Split File option. 2. Generate scatterplots between the dependent variable and each of the covariates. Procedure for checking linearity for each group STEP 1 1. Click on Graphs, then click on Scatter. 2. Click on Simple. Click on the Define button. 3. In the Y axis box put your dependent variable (SF-36 (physical dimension) score six months post-operation) 4. In the X axis box put your covariate (SF-36 (physical dimension) score preoperation) 5. In the Set markers by box put your independent variable (eg. type.op). Click on OK. STEP 2 1. Once you have your scatterplot displayed in the Viewer window, double click on it. This opens the Chart Editor window. 2. Click on Chart from the top menu, then Options. 3. In the Display options section make sure that there is a tick in the Show subgroups box. 4. In the Fit Lines section tick Subgroups. 5. Click on the Fit Options button. In the Regression Options section tick the Display R square in legend. 6. Click on Continue and then OK. 7. To close the graph and return to your normal Viewer window, just click on File from the menu and then Close.
100
80
Type of Operation
60 Phaco Rsq = 0.9720 40 thru origin ECCE Rsq = 0.9543 20 0 20 40 60 80 100 120 thru origin
PHYS4
PHYS1
The output generated using the procedures detailed above provides you with a number of useful pieces of information. First, you can look at the general distribution of scores for each of your groups. Does there appear to be a linear (straight-line) relationship for each group. What you dont want to see is an indication of a curvilinear relationship. In the above example the relationship is clearly linear, so we have not violated the assumption of a linear relationship. If you find a curvilinear relationship you may want to reconsider the use of this covariate, or alternatively you could try transforming the variable and repeating the scatterplot to see if there is an improvement. The R squared values, which are given in the legend for each group, gives you an indication of the strength of the relationship between your dependent variable (SF-36 (physical dimension) score six months post-operation) and your covariate (SF-36 (physical dimension) score pre-operation). In this case these two variables are strongly correlated. For the ECCE group, 95% of the variance in scores at 6 months postoperation are explained by scores at pre-operation. To get this value, just convert R squared value to a percentage by shifting the decimal point two places). For the Phaco group 97% of the variance in the two sets of scores are shared.
45
In many studies, particularly when you are using different scales as covariates, you will not find such high correlations. The high correlation here is due in part to the preintervention, post-intervention design, which used the same test administered on two different occasions. If you find in your study that there is only a very weak relationship between your dependent variables and the covariate you may need to consider using alternative covariates. There is no point controlling for a variable, if it is not related to the dependent variable. Assumption 5: Homogeneity of regression slopes This final assumption (homogeneity of regression slopes) concerns the relationship between the covariate and the dependent variable for each of your groups. What you are checking is that there is no interaction between the covariate and the treatment or experimental manipulation. There are a number of different ways to evaluate this assumption. Graphically, you can inspect the scatterplot between the dependent variable and the covariate obtained when testing for Assumption 4. Are the two lines (correspond to the two groups in this study) similar in their slopes? In the above example the two lines are very similar, therefore it does not appear that we have violated this assumption. If the lines had been noticeably different in their orientation, this may suggest that there is an interaction between the covariate and the treatment (as shown by the different groups). This would mean a violation of this assumption. This assumption can also be assessed statistically, rather than graphically. This involves checking to see whether there is a statistically significant interaction between the treatment and the covariate. If the interaction is significant at alpha level of 0.05, then we have violated the assumption. The procedure to check is described next. Procedure to check for homogeneity of regression slopes 1. From the menu at the top of the screen click on: Analyze General Linear Model Univariate 2. In the Dependent Variables box put your dependent variable (SF-36 (physical dimension) score six months post-operation). 3. In the Fixed Factor box put your independent or grouping variable (type of operation, type.op). 4. In the Covariate box put your covariate (SF-36 (physical dimension) score preoperation) 5. Click on the Model button. Click on Custom. 6. Check that the Interaction option is showing in the Build Terms box.
46 7. Click on your independent variable (type.op) and then the arrow button to move it into the Model box. 8. Click on your covariate (phy1) and then the arrow button to move it into the Model box. 9. Go back and click on your independent variable (type.op) again on your left-hand side (in the Factors and Covariates section). While this is highlighted, hold down the Ctrl button on your keyboard, and then click on your covariate variable (phy1). Click on the arrow button to move this into the right-hand side box labeled Model. 10. In the Model box you should now have listed: - your independent variable (type.op) - your covariate (phy1); and - an extra line of the form: covariate*independent variable (phy1*type.op) 11. The final term is the interaction that we are checking for. 12. Click on Continue and then OK. The output generated by this procedure is shown below.
Descriptive Statistics Dependent Variable: PHYS4 Type of Operation ECCE Phaco Total Mean 84.3000 86.6000 85.4500 Std. Deviation 17.3208 16.2707 16.7588 N 50 50 100
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+TYPE.OP+PHYS1+TYPE.OP * PHYS1
47
a . C o m p u te d u s in g a lp h a = .0 5 b . R S q u a re d = .3 7 9 (A d ju s te d R S q u a r e d = .3 6 0 )
Type of Operation Dependent Variable: PHYS4 Type of Operation ECCE Phaco Mean Std. Error 84.300 a 1.896 86.600 a 1.896 95% Confidence Interval Lower Bound Upper Bound 80.536 88.064 82.836 90.364
In the output obtained from this procedure that only value that you are interested in is the significance level of the interaction term (phy1*type.op). You can ignore the rest of the output. If the Sig. level for the interaction is less than 0.05, then youre your interaction is statistically significant, indicating that you have violated the assumption. In this situation we do not want a significant result. We want a Sig. value of greater than 0.05. In the above example the Sig. level is 0.370, safely above the cut-off. We have not violated the assumption of homogeneity of regression slopes. This support the earlier conclusion gained from an inspection of the scatterplots for each group. Now that we have finished checking the assumptions, we can proceed with the ANCOVA analysis to explore the differences between our treatment groups.
48 Procedure for one-way ANCOVA 1. From the menu at the top of the screen click on: Analyze General Linear Model Univariate 2. In the Dependent Variables box put your dependent variable (phy4). 3. In the Fixed Factor box put your independent or grouping variable (type of operation, type.op). 4. In the Covariate box put your covariate (phy1) 5. Click on the Model button. Click on Full Factorial in the Specify Model section. Click on Continue. 6. Click on the Options button. 7. In the top section labeled Estimated Marginal Means click on your independent variable (type.op). 8. Click on the arrow button to move it into the box labeled Display Means for. This will provide you with the mean score on your dependent variable for each group, adjusted for the influence of the covariate. 9. In the bottom section of the Options dialogue box choose: a. Descriptive Statistics b. Estimated of Effect Size c. Observed Power d. Homogeneity tests 10. Click on Continue and then OK. The output generated by this procedure is shown below.
Descriptive Statistics Dependent Variable: PHYS4 Type of Operation ECCE Phaco Total Mean 84.3000 86.6000 85.4500 Std. Deviation 17.3208 16.2707 16.7588 N 50 50 100
49
a Levene's Test of Equality of Error Variances
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+PHYS1+TYPE.OP
T e s ts o f B e tw e e n -S u b je c ts Effe c ts D e p e n d e n t V a ria b le : P H Y S 4 T yp e III S u m S o u rce o f S q u a re s C o rre cte d M o d e l1 0 3 9 6 .4 3b1 In te rce p t 1 0 4 3 9 .6 1 3 PHYS1 1 0 2 6 4 .1 8 1 T Y PE .O P 1 3 2 .2 5 0 E rro r 1 7 4 0 8 .3 1 9 T o ta l 7 5 7 9 7 5 .0 0 0 C o rre cte d T o ta l 2 7 8 0 4 .7 5 0 df 2 1 1 1 97 100 99 M e a n Sq u a re F 5 1 9 8 .2 1 5 2 8 .9 6 5 1 0 4 3 9 .6 1 3 5 8 .1 7 0 1 0 2 6 4 .1 8 1 5 7 .1 9 3 1 3 2 .2 5 0 .7 3 7 1 7 9 .4 6 7 N o n c e n t. O b se rv e d a S ig . Eta S q u a re d P a ra m e te r P o w e r .0 0 0 .3 7 4 5 7 .9 2 9 1 .0 0 0 .0 0 0 .3 7 5 5 8 .1 7 0 1 .0 0 0 .0 0 0 .3 7 1 5 7 .1 9 3 1 .0 0 0 .3 9 3 .0 0 8 .7 3 7 .1 3 6
a . C o m p u te d u sin g a lp h a = .0 5 b . R Sq u a re d = .3 7 4 (A d ju ste d R S q u a re d = .3 6 1 )
50 INTERPRETATION OF OUTPUT FROM ONE-WAY ANCOVA There are a number of steps in interpreting the output from ANCOVA. 1. In the box labeled Descriptive Statistics, check that the details are correct (eg. number in each group, mean scores). 2. The details in the table labeled Levenes Test of Equality of Error Variances allow you to check that you have not violated the assumption of equality of variance. You want the Sig. value to be greater than 0.05. If this value is smaller than 0.05 (and therefore significant) this means that your variance are not equal, and that you have violated the assumption. In this case we have not violated the assumption because our Sig. value is 0.408, which is much larger than 0.05. 3. The main ANCOVA results are presented in the table labeled Test Of BetweenSubjects Effects. We want to know if our groups are significantly different in terms of their scores on the dependent variable (phy4). Find the line corresponding to your independent variable (type.op) and read across to the column labeled Sig. If the value in this column is less than 0.05, then your groups differ significantly. In this case our value is 0.393, therefore is not significant. There is no significant difference in the quality of life (physical dimension) for subjects in the ECCE group and Phaco group, after controlling for scores on the quality of life (physical dimension) administered prior to the operation. 4. You should also consider the effect size, as indicated by the corresponding eta squared value. The value in this case is only 0.008 (a small effect size). This value also indicates how much of the variance in the dependent variable is explained by the independent variable. Convert the eta squared value to a percentage by multiplying by 100. In this example, we are only able to explain 0.8 percent of the variance. 5. The other piece of information that we can gain from this table concerns the influence of our covariate. Find the line in the table which corresponds to the covariate (phy1). Read across to the Sig. level. This indicates whether there is a significant relationship between the covariate and the dependent variable, while controlling the independent variable (type.op). In the line corresponding to covariate (phy1) you will see that the Sig. value is 0.000 (which actually means less than 0.0005) and therefore our covariate is significant. In fact it explained 37% of the variance in the dependent variable (eta squared of 0.371 multiplied by 100). 6. The final table in the ANCOVA output (Estimated marginal means) provides us with the adjusted means on the dependent variable for each of our groups. Adjusted refers to the fact that the effect of the covariate has been statistically removed.
51 PRESENTING THE RESULTS FROM ONE-WAY ANCOVA The result of this one-way analysis of covariance could be presented as follows: One-way between-groups analysis of covariance was conducted to compare the effectiveness of two different operation performed to treat cataract. The independent variable was the type of operation (ECCE and Phaco) and the dependent variable consisted of scores on quality of life (physical dimension) administered after six months operation. Patients scores on the pre-operation administration of the quality of life (physical dimension) were used as the covariate in this analysis. Preliminary checks were conducted to ensure that there was no violation of the assumptions of normality, linearity, homogeneity of variance, homogeneity of regression slopes, and reliable measurement of the covariate. After adjusting for pre-operation scores, there was no significant difference between the two operation groups on post-operation scores on quality of life (physical dimension), F = 0.737, p = 0.393, eta squared = 0.008. There was a relationship between pre-operation and post-operation scores on the quality of life (physical domain), as indicated by an eta squared value of 0.371. In presenting the results of this analysis you also provide a table of mean for each of the groups. If the same scale is used to measure the dependent variable (phys4) and the covariate (phys1), then you would include the means at phys1 and phys4. You can get these by running Descriptives. If the different scale is used to measure the covariate you would provide the unadjusted mean (and standard deviation), and the adjusted mean (and standard error) for the two groups. The unadjusted mean is available from the Descriptive statistics table. The adjusted mean (controlling for the covariate) is provided in the Estimated marginal means table. It is also a good idea to include the number of cases in each of your groups.
52 TWO-WAY ANCOVA Two-way ANCOVA involves 2 independent, categorical variables (with 2 or more levels or conditions), one dependent continuous variable, and one / more continuous covariates. The example that will be used here is an extension of that presented in the one-way ANCOVA. In that analysis we were interested in determining which operation (ECCE or Phaco) was more effective in increasing the quality of life (physical dimension). We found that no significant difference between the two groups. Suppose that in reading further in the literature, some research suggested there might be a difference in levels of education respond to different operations. Sometimes in the literature you will see this additional variable described as a moderator. This is, it moderates or influences the effect of the other independent variable. Often these moderator variables are individual difference variables, characteristics of individuals that influence the way in which they respond to an experimental manipulation or treatment condition. It is important if you obtain a non-significant result for your one way ANCOVA that you consider the possibility of moderator variables. In your own research always consider factors such as gender, age and level of education, as these can play an important part in influencing the results. Research on males (or higher level of education) sometimes yield quite different results when repeated using female (or lower level of education) samples. The message here is to consider the possibility of moderator variables in your research design and, where appropriate, include then in your study. Details of example In the example presented below we will use the same data with an additional independent variable (level of education). This will allow us to broaden the analysis to see if level of education is acting as a moderator variable in influencing the effectiveness of the two operations. We are interested in the possibility of a significant interaction effect between level of education and operation group. File CIS Variable Variable label type.op Type of operation educatio Level of education phys1 SF-36 (physical dimension) scores preoperation SF-36 (physical dimension) scores 6 months post-operation Coding instructions 1 = ECCE 2 = Phacoemulsification (Phaco) 1 = Primary / No schooling 2 = Secondary 3 = Tertiary Total scores on the quality of life (physical dimension) administered prior to the operation. Scores range from 0 100. High scores indicate higher levels of quality of life. Total scores on the quality of life (physical dimension) administered 6 months after the operation. Scores range from 0 100. High scores indicate higher levels of quality of life.
phys4
53
Summary for two-way ANCOVA Research question: Does levels of education influence the effectiveness of two type of cataract operations (ECCE and Phaco) ? Is there a significant difference in post-operation quality of life (physical dimension) between three levels of education in their response after cataract operation ? At least four variables are involved: 1. Two categorical independent variable with two or more levels (levels of education: 1 = Primary / No Schooling, 2 = Secondary, 3 = Tertiary; types of operation: ECCE / Phaco) 2. One continuous dependent variable (scores on quality of life (physical dimension) at six months post-operation phys4) 3. One or more continuous covariates (scores on quality of life (physical dimension) at pre-operation phys1) ANCOVA will control for scores on your covariate(s) and then perform a normal twoway ANOVA. This will tell you if there is: 1. A significant main effect for your first independent variable (type.op) 2. A main effect for your second independent variable (level of education) 3. Whether there is a significant interaction between the two. Assumptions: All normal two-way ANOVA assumptions apply (eg. normality, homogeneity of variance). This should be checked first. Additional ANCOVA assumptions are similar with the one-way ANCOVA. Procedure for two-way ANCOVA 1. From the menu at the top of the screen click on: Analyze General Linear Model Univariate 2. In the Dependent Variables box put your dependent variable (phys4).
54 3. In the Fixed Factor box put your two independent or grouping variables (eg. level of education educatio, and type of operation type.op ). 4. In the Covariate box put your covariate(s) (phys1) 5. Click on the Model button. Click on Full Factorial in the Specify Model section. Click on Continue. 6. Click on the Options button. 7. In the top section labeled Estimated Marginal Means click on your first independent variable (eg. educatio). Click on the arrow to move it into the box labeled Display Means for. 8. Repeat for the second independent variable (eg. type.op) 9. Click on the extra interaction term included (eg. type.op*educatio). Move this into the box. This will provide you with the mean scores on your dependent variable split for each group, adjusted for the influence of the covariate. 10. In the bottom section of the Options dialogue box click on: a. Descriptive Statistics b. Estimated of Effect Size c. Observed Power d. Homogeneity tests 11. Click on Continue and then OK.
55
Descriptive Statistics Dependent Variable: PHYS4 Type of Operation Level of Education ECCE Primary Secondary Tertiery Total Phaco Primary Secondary Tertiery Total Total Primary Secondary Tertiery Total Mean 69.5833 88.6667 90.0000 84.3000 85.2941 87.2917 87.2222 86.6000 78.7931 88.0556 88.5294 85.4500 Std. Deviation 23.9752 12.1011 10.0000 17.3208 18.9135 15.8100 13.4887 16.2707 22.1865 13.7504 11.6946 16.7588 N 12 30 8 50 17 24 9 50 29 54 17 100
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Design: Intercept+PHYS1+TYPE.OP+EDUCATIO+TYPE.OP * EDUCATIO T e s t s o f B e t w e e n - S u b je c t s E f f e c t s D e p e n d e n t V a r ia b le : P H Y S 4 T y p e III S u m S o u rce o f S qu a re s b C o r r e c te d M o d e l 1 2 8 3 2 .6 2 7 In te r c e p t 1 1 0 3 7 .9 0 0 PHYS1 9 2 2 5 .5 0 3 T Y P E .O P 2 7 0 .8 3 8 E D U C A T IO 1 5 6 9 .7 8 1 T Y P E .O P * E D U C A T IO1 3 3 .4 6 5 1 E rro r 1 4 9 7 2 .1 2 3 T o ta l 7 5 7 9 7 5 .0 0 0 C o r r e c te d T o ta l 2 7 8 0 4 .7 5 0 df 6 1 1 1 2 2 93 100 99 M e a n S q u a re F 2 1 3 8 .7 7 1 1 3 .2 8 5 1 1 0 3 7 .9 0 0 6 8 .5 6 2 9 2 2 5 .5 0 3 5 7 .3 0 5 2 7 0 .8 3 8 1 .6 8 2 7 8 4 .8 9 0 4 .8 7 5 5 6 6 .7 3 3 3 .5 2 0 1 6 0 .9 9 1 N o n c e n t. O b s e r v e d a S ig . E ta S q u a r e dP a r a m e te r P o w e r .0 0 0 .4 6 2 7 9 .7 1 0 1 .0 0 0 .0 0 0 .4 2 4 6 8 .5 6 2 1 .0 0 0 .0 0 0 .3 8 1 5 7 .3 0 5 1 .0 0 0 .1 9 8 .0 1 8 1 .6 8 2 .2 5 0 .0 1 0 .0 9 5 9 .7 5 1 .7 9 1 .0 3 4 .0 7 0 7 .0 4 1 .6 4 3
a . C o m p u te d u s in g a lp h a = .0 5 b . R S q u a r e d = .4 6 2 ( A d ju s te d R S q u a r e d = .4 2 7 )
56
2. Level of Education Dependent Variable: PHYS4 Level of Education Primary Secondary Tertiery Mean Std. Error 78.854 a 2.399 87.223 a 1.740 a 88.883 3.083 95% Confidence Interval Lower Bound Upper Bound 74.089 83.618 83.767 90.679 82.761 95.005
3. Type of Operation * Level of Education Dependent Variable: PHYS4 Type of Operation Level of Education ECCE Primary Secondary Tertiery Phaco Primary Secondary Tertiery Mean Std. Error 71.962 a 3.676 a 87.809 2.319 89.648 a 4.486 a 85.745 3.078 86.636 a 2.591 88.118 a 4.231 95% Confidence Interval Lower Bound Upper Bound 64.662 79.262 83.203 92.415 80.739 98.557 79.633 91.857 81.490 91.782 79.716 96.520
57
INTERPRETATION OF OUTPUT FROM TWO-WAY ANCOVA There are a number of steps in interpreting the output from a two-way ANCOVA. 1. In the box labeled Descriptive Statistics, check that the details are correct (eg. number in each group, mean scores). 2. The details in the table labeled Levenes Test of Equality of Error Variances allow you to check that you have not violated the assumption of equality of variance. You want the Sig. value to be greater than 0.05. If this value is smaller than 0.05 (and therefore significant) this means that your variance are not equal, and that you have violated the assumption. 3. The main ANCOVA results are presented in the table labeled Test Of BetweenSubjects Effects. We want to know if there is a significant main effect for any of our independent variables (type.op or level of education) and whether the interaction between these two variables is significant. Of most interest is the interaction, so we will check this first. If the interaction is significant, then the two main effects are not important, because the effect of one independent variable is dependent on the level of the other independent variable. Find the line corresponding to the interaction effect (type.op*educatio). Read across to the column labeled Sig. If the value in this column is less than 0.05, then the interaction is significant. In this case our value is 0.034, therefore our result is significant. This significant interaction effect suggests that patients respond differently to the two operation based on their level of education. The main effects for level of education is significant (p = 0.010) but for type of operation is not statistically significant (p = 0.198). 4. You should also consider the effect size, as indicated by the corresponding eta squared value. The value in this case is 0.070 (moderate to large effect size). This value also indicates how much of the variance in the dependent variable is explained by the independent variable. Convert the eta squared value to a percentage by multiplying by 100. In this example, we are able to explain 7 percent of the variance. 5. The other piece of information that we can gain from this table concerns the influence of our covariate. Find the line in the table which corresponds to the covariate (phys1). Read across to the Sig. level. This indicates whether there is a significant relationship between the covariate and the dependent variable, while controlling the independent variable (type.op). In the line corresponding to covariate (phys1) you will see that the Sig. value is 0.000 (which actually means less than 0.0005) and therefore our covariate is significant. In fact it explained 38.1% of the variance in the dependent variable (eta squared of 0.381 multiplied by 100). 6. The final table in the ANCOVA output (Estimated marginal means) provides us with the adjusted means on the dependent variable for each of our groups, split according to each of our independent variables separately and then jointly. Adjusted refers to the fact that the effect of the covariate has been statistically removed.
58
PRESENTING THE RESULTS FROM TWO-WAY ANCOVA The result of this two-way analysis of covariance could be presented as follows: A 2 by 2 between-groups analysis of covariance was conducted to assess the effectiveness of two different operation performed to treat cataract. The independent variable was the type of operation (ECCE and Phaco) and level of education. The dependent variable consisted of scores on quality of life (physical dimension) administered after six months operation. Patients scores on the preoperation administration of the quality of life (physical dimension) were used as the covariate to control for individual differences. Preliminary checks were conducted to ensure that there was no violation of the assumptions of normality, linearity, homogeneity of variance, homogeneity of regression slopes, and reliable measurement of the covariate. After adjusting for pre-operation scores, there was a significant interaction effect, F = 3.520, p = 0.034, with a moderate to large effect size (eta squared = 0.070). The main effects for level of education was statistically significant (F = 4.875, p = 0.010), but for type of operation was not significant (F = 1.682, p = 0.198). These results suggest that patients respond differently to the two operation based on their level of education. In presenting the results of this analysis you also provide a table of means for each of the groups. If the same scale is used to measure the dependent variable (phys4) and the covariate (phys1), then you would include the means at phys1 and phys4. You can get these by running Descriptives. If the different scale is used to measure the covariate you would provide the unadjusted mean (and standard deviation), and the adjusted mean (and standard error) for the two groups. The unadjusted mean is available from the Descriptive statistics table. The adjusted mean (controlling for the covariate) is provided in the Estimated marginal means table. It is also a good idea to include the number of cases in each of your groups.