Not all evidence is equal: from lab dish to clinical trial

“Shown to boost collagen production by 300%.” Sounds impressive, until you learn the test was done on cells in a plastic dish. Evidence in skin care comes in rungs, and a claim that’s true on one rung can be meaningless on another. Here’s the ladder, from the bottom up, and what each step can and can’t tell you about your own skin.

Why there’s a ladder at all

Researchers rank evidence because some study designs are more prone to fooling you than others. A 2011 review in Plastic and Reconstructive Surgery traces the idea to a 1979 Canadian task force and describes the systems used since: at the top, large randomized trials and reviews that pool them; at the bottom, case series with no controls and expert opinion.

Lower rungs are where ideas start. The problem is only when a claim borrows the confidence of a high rung while standing on a low one.

Rung 1: the lab dish (in vitro)

In vitro means “in glass”. The ingredient is applied to cells grown in a dish, to a slice of donated skin, or to a lab-made skin substitute, and something is measured: did the cells make more of a protein, did the ingredient pass through the sample?

This is useful, cheap and ethical, and it’s where most ingredient stories begin. But it answers a narrow question. A 2023 review of in vitro skin penetration testing in Frontiers in Pharmacology explains that these systems are built to “emulate the in vivo environment” and have improved a lot, but also lists what they miss: the skin’s own metabolism (living skin can chemically change an ingredient after it lands), and the full complexity of the skin barrier, which the authors note is “difficult to reproduce in vitro.”

So a dish result tells you an ingredient can do something to cells that it touches. It doesn’t tell you whether the ingredient in a finished cream, at that concentration, gets through the stratum corneum (the dead outer layer that keeps most things out) in enough quantity to matter, or whether it survives the trip.

“In laboratory tests”, “on skin cells”, a percentage with no people attached: that’s rung one.

Rung 2: animals

Animal studies put the ingredient into a living body, which adds biology a dish can’t. But it isn’t your biology.

A 2018 paper in the Journal of Translational Medicine argues the point bluntly: even if decades were spent improving animal models, “species differences would continue to make extrapolation from animals to humans unreliable.” A 2019 scoping review in the same journal collected studies that had actually compared animal results with human results and found reported agreement rates spanning “from 0% to 100%”, which the authors take to mean that success is hard to predict in advance.

For cosmetics there’s a Canadian twist. Since December 22, 2023, Health Canada prohibits testing cosmetics on animals in Canada and selling a cosmetic whose safety relies on new animal data, and companies must have evidence for any “not tested on animals” claim. So you’ll rarely see animal data behind a moisturizer claim, except for individual ingredients studied years ago for other purposes.

Rung 3: small open-label studies on people

Now we’re on human skin, a real step up. An open-label study means everyone knows who got what, and often everyone got the product. Twenty people use a cream for four weeks and their skin is measured before and after.

This is where most “clinically tested” claims live. What it lacks is a comparison. Skin changes with the seasons, with attention, with the act of rubbing anything in twice a day. Without a group that used a plain cream, you can’t separate the product from the calendar. And with 20 people, chance alone can produce a “significant” result.

The 2011 review puts uncontrolled case series at the lowest level of its hierarchy. That doesn’t make them worthless; it makes them a reason to do a better study.

Rung 4: the randomized controlled trial

A randomized controlled trial (RCT) adds the two things rung three was missing: a comparison group, and a coin flip to decide who lands in it. Blinding, where neither participants nor assessors know who got what, closes the last gap. This is the design that can show the product caused the change.

Two caveats, both from the reviews above. First, the 2011 review warns that the label doesn’t guarantee the quality: “poorly designed RCTs have the same level of evidence as a cohort study” in quality-adjusted systems, and a small trial can miss a real effect. Second, a 2014 review of evidence-based dermatology by a professor of dermato-epidemiology notes that RCTs are “not a good source for picking up rare but important adverse effects,” because they’re too short and too small to catch something that happens to one person in a thousand.

For a cosmetic, an RCT is unusual and a strong sign the company took the question seriously. Look for the number of people, the length, what was measured and whether it was blinded.

Rung 5: the systematic review

A systematic review is a study of studies. Researchers write a plan in advance, search for every trial on a question, judge each one’s quality, and pool the results where the trials are similar enough. The 2014 dermatology review explains why this sits at the top: systematic reviews “use explicit methods” and “seek to include all relevant studies rather than ones selected on the basis of their results or how easy they are to find.”

Its example is sobering. Seven small trials of a treatment for women in premature labour looked “conflicting” for years; pooled in 1989, the treatment turned out to be clearly effective, and the delay had cost lives. At lower stakes, the same logic applies to skin care: five small studies that disagree may, combined, point one way.

The catch: a review is only as good as the studies it can find. If every trial of an ingredient was small, short and manufacturer-run, the review will say so; “insufficient evidence” is a legitimate finding.

Where skin care claims usually sit

Most claims on non-prescription skin care come from rungs one and three: a dish result about an ingredient, plus a small open-label test of the finished product. That’s not fraud; it’s the norm for a category where products are meant to cleanse, moisturize and improve appearance, not treat disease. Here’s how to place a claim:

• “Boosts”, “stimulates”, “activates” + a cell process (collagen, cell turnover): almost always rung one.
• A percentage with no people or timeframe attached: rung one or a very small rung three.
• “Clinically tested”, “dermatologist tested”: rung three, usually a safety or irritation test rather than a performance test.
• “In a randomized, controlled study of N people over N weeks”: rung four. Rare, and worth noticing.
• A citation to a published systematic review: rung five. Rarer still; check that the review is about the product, not just an ingredient.

Ingredients with real rung-four and rung-five evidence exist, mostly ones studied as medicines. For an ingredient only ever tested in a dish, the fair statement is “this might do something; nobody has checked on skin.” If you’re choosing products to manage a skin condition, eczema (atopic dermatitis, the itchy inflamed rash), acne or rosacea, that uncertainty matters more. If it isn’t better after two weeks, see your doctor or pharmacist.

What to remember: the ladder runs dish, animal, small open-label study, randomized controlled trial, systematic review. Each rung answers a different question, and only the top two can tell you a product caused an effect on real skin. When you read a claim, find the rung it’s standing on, and don’t let a rung-one result speak with a rung-four voice.