Case - Can This Survey Be Saved?

Question

"What's troubling me is that you can't just pick a new random sample just because somebody didn't like the results of the first survey. Please tell me more about what's been done." Your voice is clear and steady, trying to discover what actually happened and, hopefully, to identify some useful information without the additional expense of a new survey.
"It's not that we didn't like the results of the first survey," responded Steegmans, "it's that only 54% of the membership responded. We hadn't even looked at their planned spending when the decision [to sample again] was made. Since we had (naively) planned on receiving answers from nearly all of the 400 people initially selected, we chose 200 more at random and surveyed them also. That's the second sample." At this point, sensing that there's more to the story, you simply respond "Uh huh . . ." Sure enough, more follows:
"Then Eldredge had this great idea of following up on those who didn't respond. We sent them another whole questionnaire, together with a crisp dollar and a letter telling them how important their responses are to the planning of the industry. Worked pretty well. Then, of course, we had to follow up the second sample as well."
"Let me see if I understand," you reply. "You have two samples: one of 400 people and one of 200. For each, you have the initial responses and followup responses. Is that it?"
"Well, yes, but there was also the pilot study - 12 people in offices downstairs and across the street. We'd like to include them with the rest because we worked so hard on that at the start, and it seems a shame to throw them away. But all we really want is to know average spending to within about a hundred dollars."
At this point, you feel that you have enough of the background information to evaluate the situation and to either recommend an estimate or an additional survey. Additional details for the survey of the 8,391 overall membership in order to determine planned spending over the next quarter are provided on the following page.
Discussion Questions
1. Do you agree that drawing a second sample was a good idea?
2. Do you agree that the followup mailings were a good idea?
3. How might you explain differences among averages in the results?
4. Should the pilot data be included? Why or why not?
5. Are any or all of the results here useful? How would you treat the data that has been collected? What additional data collection, if any, would you do?

From: Siegel, A.F. (1997). Practical Business Statistics, 3rd Edition. Irwin/McGrawHill.
�

Pilot
Study

First Sample

Second Sample

Both Samples

All Combined

Initial Mailing

Mailed

12

400

200

600

612

Responses

12

216

120

336

348

Average

$39,274.89

$3,949.40

$3,795.55

$3,894.45

$5,114.47

Std. Dev.

$9,061.91

$849.26

$868.39

$858.02

$6,716.42

Followup Mailing

Mailed

0

184

80

264

264

Responses

0

64

18

82

82

Average

$1,238.34

$1,262.34

$1,243.60

$1,243.60

Std. Dev.

$153.19

$156.59

$153.29

$153.29

Initial and Followup Mailings Combined

Mailed

12

400

200

600

612

Responses

12

280

138

418

430

Average

$39,274.89

$3,329.73

$3,465.13

$3,374.43

$4,376.30

Std. Dev.

$9,061.91

$1,364.45

$1,179.50

$1,306.42

$6,229.77