Why the number of people in a skin study matters

“Tested on 20 people” and “tested on 1,400 people” can both appear in the small print, and they don’t carry the same weight. The number of people in a study decides how much luck is mixed into the result. Here’s why, with a real example from eczema research where a small trial and a large one gave very different answers.

Small studies are noisy

Skin varies a lot from person to person, and from week to week in the same person. Winter hits, a new detergent appears, stress flares an itch. In a small group, a few people having an unusually good or bad month can swing the average.

That noise cuts both ways:

• A small study can find a big effect that isn’t real. A few lucky participants in the product group are enough.
• A small study can miss an effect that is real. The signal gets lost in the noise.

Small studies are very common in skin research. The CONSORT guidance, an international standard for reporting clinical trials published in the BMJ, cites a review that found the median dermatology trial had just 46 patients, across 73 trials. That isn’t automatically bad. Early studies are supposed to be small. It just means one small study is a hint, not an answer.

A real example: the small trial and the big one

Here’s a story eczema researchers tell often. Eczema (atopic dermatitis, the itchy, inflamed rash) often starts in babies, and a weak skin barrier seems to come first. So researchers asked: if parents moisturize a newborn every day, does that prevent eczema?

The small trial

In 2014, a randomized trial in the US and UK enrolled 124 newborns at high risk of eczema. Half got daily full-body moisturizer, half didn’t. By six months, the moisturized group had about half the risk of eczema. The result was statistically significant, and it made headlines.

But look at the uncertainty. The study reported the risk in the moisturized group as somewhere between 28% and 90% of the risk in the other group. That’s a very wide range: anything from a huge benefit to a modest one. To their credit, the authors said plainly that the idea needed to be “confirmed in larger trials”. The trial’s first job was to check whether families would even sign up.

The big trial

In 2020, the BEEP trial in the UK put that idea to a proper test. It enrolled 1,394 newborns with a family history of eczema, asthma or hay fever. At age two, eczema was present in 23% of the moisturized group and 25% of the control group. That difference was small enough to be chance. The range of plausible effects was also much narrower than in the small trial. The moisturized group also had slightly more skin infections.

Then a Cochrane review in 2022 (Cochrane reviews pool all the good trials on a question) combined 7 trials with 3,075 babies. Its conclusion: moisturizing healthy babies in the first year “probably” doesn’t change the risk of eczema.

The small trial wasn’t dishonest. It was just small. This is how science is supposed to work, and it’s a good reason to be cautious about any skin care claim built on one small study.

What “statistical power” means

Power is the chance that a study will catch a real difference if there is one. The CONSORT guidance puts it this way: a study should be large enough to have a high probability of detecting, as statistically significant, a difference that actually matters.

Before a good trial starts, researchers estimate how big a difference they expect and how much results usually vary. Then they calculate how many people they need. Power is commonly set at 80% or 90%. A cosmetic study that never did this calculation may have recruited however many people were convenient.

Low power also causes a quieter mistake. CONSORT warns that small studies often wrongly conclude the groups “do not differ, when in fact too few patients were studied to make such a claim.” “No difference found” in a tiny study doesn’t mean “no difference exists”.

Dropouts: count who finished

A study that starts with 200 people might end with 150. Who left matters.

People don’t drop out at random. Someone whose cream stung, or who saw no change, is more likely to stop showing up than someone who loved it. If the analysis only includes the people who stayed, the product can look better than it is.

That’s why CONSORT asks trials to report, for each group, how many people were lost after they were assigned to a group, and why. In BEEP, for example, 1,394 babies were randomized and outcome data were available for 1,210. The researchers analyzed babies in the group they were assigned to, whether or not parents stuck with the moisturizer, which keeps the comparison fair. Adherence was not perfect either: among families who answered the questionnaires, 74% were still moisturizing at 12 months.

If a skin care claim says “tested on 60 people” but the results are based on 41, ask what happened to the other 19.

Subgroups: the result hiding in a slice

Sometimes a product doesn’t beat the comparison overall, but a result is found in a slice of the group: “in women over 40”, “in people with very dry skin”. Be careful here.

Each slice is smaller than the whole study, so it’s noisier. And the more slices you check, the more likely one of them looks impressive by pure chance. CONSORT says subgroup analyses carry a “high risk for spurious findings”, and that those chosen after looking at the data “are especially likely not to be confirmed by further studies.”

The Cochrane review did it the careful way: it planned its subgroups in advance (age, family risk, a gene linked to eczema, and others) and found none of them changed the result.

Advertising has its own version of this rule. Ad Standards, Canada’s industry self-regulator, has guidelines for surveys that back comparative ad claims. They recommend a sample of at least 300 for claims about a large population, and at least 100 people in any subgroup the ad specifically mentions. Those guidelines cover surveys rather than skin trials, but the logic is the same: a claim about a slice needs enough people in that slice.

What to look for

When a claim quotes a study, check:

• How many people? No number usually means a small one.
• Is it one study or several? One small positive study is a starting point.
• How wide is the range? If the study reports a range of plausible effects, a very wide one means lots of uncertainty.
• How many finished? Compare the number who started with the number analyzed.
• Is the headline from everyone, or a slice? Results in one subgroup need confirming.

And if you or your child has a skin condition, a study’s size is only part of the picture. If eczema or a rash isn’t better after two weeks of gentle care, see your doctor or pharmacist.

What to remember: more people means less luck in the result. Small studies are useful first steps, but they can overstate or miss effects, as the small and large eczema-prevention trials showed. Check how many people started, how many finished, and whether the good news comes from everyone or from a hand-picked slice.