Engineering & Technology · Engineering Design Methods & Trade-offs

Prototypes, testing and learning from unsuccessful designs

Two carrier grips look plausible on paper. A quick foam mock-up can reveal whether fingers have space, but it cannot establish the strength of the eventual joint. Building the most realistic-looking prototype is not always the fastest way to learn the thing that could change a decision.

Use the opening design, measurement, sketching and everyday-materials ideas as support. This chapter introduces its own comparison tools and explains the simple weighted sums and cost equations when needed. All users, observations, test results, prices, dimensions and energy values are constructed teaching cases. None is a product specification, legal compliance assessment, real study or population capability limit.

Back to the chapter

After reading, you can

Choose prototypes and tests that can distinguish an important uncertainty, and preserve learning from failure.

Build a prototype around a learning question

A prototype is a representation made to explore or test a proposed solution. Its fidelity is how closely selected features resemble the intended result. A flat paper layout may reproduce labels; a full-size mock-up may reproduce reach and clearance; a material specimen may explore a joining process. Specify which features are faithful and which are deliberately omitted.

Start with the uncertain claim that matters: can a user identify the return position, does the handle interfere with removal, or does the proposed joint retain its function under the defined test? Choose the simplest representation that can answer that question credibly. Do not promote a successful label mock-up into evidence of structural durability.

Plan what would count as evidence before testing

Write the task, setup, sample, procedure and observations first. Include what counts as success, what counts as assistance, what is recorded when the task cannot be completed, and conditions for stopping the test. Use suitable supervision and methods for the actual task. This chapter’s cases are paper and low-load mock-up examples, not instructions for testing load-bearing equipment.

When comparing prototypes, avoid giving one an advantage through familiarity, order, wording or a different payload. Counterbalancing changes the presentation order across participants: some encounter A first, others B first. It can reduce an order effect but does not guarantee an unbiased sample or remove fatigue. Retain individual outcomes so a pooled average does not hide important differences.

The figure shows constructed paired observations for four participants. Each person tries both paper interfaces, with alternating order. Paired points let us compare within a person rather than confusing differences between people with differences between interfaces.

P1 A/B18/14 s, A first; P2 22/17, B first; P3 16/15, A first; P4 24/18, B first. A mean20 s; B mean16 s. These invented pairs do not establish population effects.

A failed test should sharpen the next question

If a mock-up fails, record the condition and the observed failure before assigning a cause. A lid that opens during movement may have a catch problem, a deformation problem or a test setup unlike intended use. Photograph or sketch the relevant geometry and retain the version. Do not quietly strengthen the catch and overwrite the failed case.

Change a suspected cause and choose a test that can discriminate it from competing explanations. Changing several features at once may improve performance but make the mechanism harder to identify. Sometimes that is acceptable for a design comparison; it is weaker evidence for a claim about which individual feature caused the improvement.

A successful small trial narrows uncertainty within its scope. It does not establish rare-event reliability, long-term wear or performance for every user. State what the prototype taught, what remains untested, and which next decision the evidence can support.

Read a paired mock-up comparison

Four invented participants complete an identification task. Their A/B times in seconds are 18/14, 22/17, 16/15 and 24/18. Each completes both tasks without assistance. A’s mean is 80/4 = 20 s; B’s is 64/4 = 16 s. The within-person reductions are 4, 5, 1 and 6 s, averaging 4 s.

Relative to the A mean, the mean reduction is 4/20 = 20 %. B is faster for all four observed pairs in this constructed trial. That supports further evaluation of B’s label arrangement; it does not prove why it helped or establish a 20 % benefit for a wider population. Record the two A-first and two B-first orders and examine whether order might still matter.

Try the reasoning

Can a foam handle mock-up establish the fatigue life of a final metal joint?
Show the answer and reasoning
No. It preserves selected geometry, not the final material, joint process or repeated-load behaviour.
Why vary which prototype a participant encounters first?
Show the answer and reasoning
Practice or fatigue can affect a second attempt. Varying order helps investigate or reduce systematic order advantage.
If three features change and performance improves, have you identified the cause?
Show the answer and reasoning
Not individually. The combined design may work better, but another comparison is needed to separate the features’ effects.

What to carry forward

Prototype a question, test it consistently, and keep failed versions so the next iteration has an evidence trail.

References

Teaching resources and research supporting this lesson’s explanations and examples.