Design surgery

Case conference and exit question

Gavin Simpson
gavin@anivet.au.dk

Aarhus University

Tuesday, 15 September 2026

Design surgery

14:15-15:45

Your task

Treat each study as a proposed test of a biological hypothesis.

  • Identify the comparison the study creates
  • Diagnose the most serious threat to inference
  • Propose the smallest defensible repair
  • State the limitation that remains

Activity timing

Time Activity
14:15-14:30 Whole-class demonstration using the pig study
14:30-15:00 Group diagnosis round
15:00-15:15 Break
15:15-15:45 Group repair round

The pig study

A researcher tests a feed additive using two pens of 12 pigs. One pen receives standard feed, while the other receives the additive. Every pig is weighed at the start and after six weeks.

The researcher plans to compare the 12 pigs in each pen.

What concerns you about the proposed comparison?

Study anatomy

Design element In this study
Biological question Does the additive affect growth?
Factor and levels Additive absent or present
Response Weight gain over six weeks
Experimental unit Pen
Treatment replication One pen per treatment
Observations 12 pigs measured twice within each pen

Main diagnosis

Diet is perfectly confounded with pen. The study has one independently treated pen per diet.

An observed difference could reflect:

  • the feed additive
  • pen environment or management
  • chance differences between these two pens

Measuring more pigs within the same pens does not separate these explanations.

Minimum repair

What is the smallest realistic change that would create independent replication of both diets?

  • Use several pens for each diet
  • Allocate pens to diets using a random process
  • Keep pen identity attached to every pig measurement
  • Estimate the diet comparison from independent pens

A repaired design

Hand-drawn experimental design with eight pens randomly allocated to standard or probiotic feed, four pens per diet, followed by pen-level comparison.

Diagnosis round

30 minutes

Work on the two cases assigned to your group.

  • Spend about 12 minutes on each case
  • Agree on one primary diagnosis
  • Record what the design does well
  • Do not redesign the experiment yet

Diagnosis record

For each case, write down:

  1. the intended biological question
  2. the treatment and response
  3. the experimental unit and true replication
  4. the comparison the design actually creates
  5. the main flaw or threat to inference

Complete this sentence: The intended conclusion is threatened because…

Repair round

30 minutes

Repair one or both cases under the printed constraint.

  • Change as little as reasonably possible
  • Preserve the original hypothesis if the resources allow it
  • Explain when the available resources cannot support the causal question
  • Draw the repaired design

Design sketch

Your drawing should make these features visible:

  • experimental units and the number assigned to each treatment
  • the method and level of treatment allocation
  • treatment and comparison conditions
  • observations or subsamples within each unit
  • the comparison used to estimate the treatment effect

A reader should be able to identify the source of independent treatment replication from the drawing.

Repair check

  • Does the repaired experiment test the original hypothesis?
  • Is each treatment independently replicated?
  • Has the main confounding or bias been removed?
  • What is the weakest remaining feature?
  • What conclusion would the repaired design support?

Case conference

15:45-15:57

Conference format

For each selected case:

  1. one group presents its diagnosis and repair
  2. another group gives a second opinion
  3. the class proposes an alternative repair
  4. we identify the trade-off and remaining limitation

Conference cases

Case Design problem to examine
E: fish diets Tanks are the experimental units. Fish are observations within tanks.
B: broiler lighting One house per lighting programme confounds lighting with house.
D: sheep supplement A before-after study lacks a contemporaneous control.

Repair trade-offs

Name the effect of each repair on:

  • risk of bias
  • precision
  • generalisability
  • logistics and cost
  • animal use

Then state the limitation that remains.

Hand-drawn scene of students comparing two alternative experimental designs for aquarium studies.

Exit question

15:57-16:00

The calf study

Twelve calves share one barn.

Six calves on the left receive milk replacer A. Six calves on the right receive milk replacer B.

Growth is measured for every calf.

Hand-drawn barn with six calves on each side receiving different milk replacers and an unknown allocation process.

Your exit question

What is the first question the researcher must answer before you can judge this design?

Write one question that would establish:

  • what receives treatment independently, or
  • how calves were allocated to milk replacers.

What the answer reveals

  • If treatment is applied independently to calves, the calf may be the experimental unit.
  • If treatment is applied to a shared group, the group is the experimental unit.
  • If replacer A is always on the left and B always on the right, treatment may be confounded with barn side.
  • The study description must report the allocation process clearly.

Sources

Ruxton, Graeme D., and Nick Colegrave. 2016. Experimental Design for the Life Sciences. 4th ed. Oxford: Oxford University Press. Chapters 1, 2, 4, 5 and 7. ISBN 978-0-19-871735-5. Publisher page.

The teaching cases and activity timings come from the course instructor guide.