Question

Jayden sets up an investigation to see how different protein shakes can impact muscle gains. He purchases two different brands of protein shakes. He drinks one brand and his friend drinks the other brand. They each perform the same exercises every day for one month. Jayden compares their muscle composition before and after the month of workouts. Evaluate Jayden’s experimental design. What type of investigation has Jayden planned: descriptive, comparative, or experimental? How could the design of Jayden’s investigation be improved? What needs to happen so that his results can become a part of the body of scientific knowledge?

Answer 1

Short answer: Jayden’s setup is an experimental/comparative investigation (he is manipulating the independent variable — brand of protein shake — and comparing outcomes), but as designed it is essentially anecdotal and invalid for scientific inference.

Why: problems with the current design
- N = 2 (Jayden and one friend) is far too small; results are anecdotal and not generalizable.
- No randomization or blinding — assignment and expectations can bias results (placebo/expectancy effects).
- Many confounding variables are not controlled: genetics, prior training status, total diet and protein intake, sleep, stress, non-exercise activity, adherence, exercise intensity/volume beyond “same exercises”.
- Short duration — 1 month may be too short to detect meaningful, reliably measurable muscle hypertrophy for many people.
- No replication, no pre-specified outcome measures or statistical plan, and likely no objective, reliable measures of muscle composition (or repeated measures to reduce error).

How to improve the design (practical, prioritized changes)
1. Increase sample size and do a power calculation to decide how many participants are needed to detect a realistic effect size.
2. Randomize participants to the two shake groups to avoid selection bias.
3. Blind participants and assessors if possible (double-blind): give shakes in identical packaging and coding so neither subjects nor the testers know which brand is which.
4. Standardize or record all major confounders:
- Prescribe the same supervised exercise program (same frequency, intensity, volume) and monitor compliance.
- Control or closely monitor total daily calories and protein intake (make shakes isocaloric and isoprotein if you want to test brand composition rather than protein amount).
- Record sleep, medication, other supplements, and prior training history.
5. Use objective, reliable outcome measures and multiple timepoints: e.g., DXA or MRI for muscle mass, ultrasound, and functional strength tests (1RM, isokinetic testing), measured by blinded assessors before and after (and possibly mid-study).
6. Consider a crossover design (each participant uses both shakes in randomized order with a sufficient washout) to reduce between-subject variability — but only if blinding and washout are feasible.
7. Lengthen study duration to a more typical hypertrophy trial (often 8–12 weeks or longer) so changes can be reliably observed.
8. Pre-register the protocol and analysis plan (hypotheses, primary/secondary outcomes, statistical tests) to avoid p-hacking.
9. Monitor adherence (e.g., supervised shakes, logs, returned packaging) and report dropouts.

What needs to happen for results to become part of scientific knowledge
- Use a rigorous, reproducible design (randomized, adequately powered, blinded if possible) and collect high-quality data.
- Analyze data with appropriate statistical methods, report effect sizes and confidence intervals, and be transparent about methods and limitations.
- Pre-register the study and share data and code when possible.
- Publish the results in a peer-reviewed journal so the work is critically evaluated and accessible.
- Independent replication: other research groups should reproduce the findings. Consistent replicated evidence is what builds the scientific consensus.

In short: Jayden’s current plan is a comparison but is essentially anecdotal. To produce scientifically useful evidence he should run a randomized, adequately powered, controlled (ideally double-blind) study with standardized protocols, objective measurements, pre-registration, and peer-reviewed publication/replication.