I have a quick question for you all. I have done a repeated measures ANOVA for hypothetical data in my Stats class. There were 5 subjects.. all subjects rated their mood on a cloudy day, then they all rated their mood on a sunny day. In sum.. they all underwent the same order of reporting mood.

Question

I found a significant effect of weather. Where I'm stuck is where my prof asked me "how sure can i be of my results?" I know drawbacks to repeated measures designs, but I'm not sure what to say beyond the usual "There could be spillover" problem. Thanks so much for any help you can offer!

My knee jerk reaction is to ask, "at what level of significance?" At the 10% level? at the 5% level? at the 1% level.

In other words, with respect to sampling "error" only, how sure are you of your results.

Your "spillover" comment suggest that your prof may be asking about non-sampling error; a very legitimate question, but hard to answer.

Thanks so much for responding. I think it was actually at the 2% level of significance.

The only thing my professor is askig is just generally "how sure can I be of my results", and he said just talk about drawbacks of the design... so I guess (hopefully) I'm on the right track.

Five subjects is a very small sample. How many measurements did you have on each subject? How did you control other factors that might effect mood (e.g., restrictions that might occur on cluody/stormy days)?

I'm not sure what you mean by "spillover," but do you know if the order of sunny and cloudy days had any effect?

Did you actually calculate the level of significance or is that just a personal estimate? ("I think it was....")

Possibly answering some of these questions will help you to estimate your sureness of your results and the conclusions they might indicate.

I hope this helps a little more. Thanks for asking.