Check before you believe: what a perfect speed score leaves out.

Why demos are fast

Headless storefronts, a fast JavaScript frontend on top of a commerce platform like Shopify, are a real and supported architecture. Demos of them are often genuinely well built. But a demo usually has mockup products, and no analytics, ad pixels, reviews, chat or testing tools. It is not a running business. So the useful question is not "is it fast?" but "fast compared with what?"

How I tested: same conditions, six runs each

I used GTmetrix, which lets you fix the test conditions so every run is comparable:

• Location: Quebec City, Canada
• Browser: Chrome, simulating an iPhone 15/16
• Connection: LTE (15/10 Mbps, 100 ms latency)
• Runs: 6 per homepage, on October 8, 2026, interleaved so no site got a "quiet" time slot

I tested three homepages: a public headless Shopify demo store, and two large, well-known brands on Shopify, one with a headless Next.js storefront and one on a Shopify theme. I'm not naming the stores, because the point is the comparison, not the brands, and any store's numbers change from week to week. A third real store timed out twice because it never stopped loading, so I left it out.

This is a small test: homepages only, one location, one day. Treat it as an example of how to check a claim, not as a ranking.

What I found

Average of 6 runs, ± standard deviation, with the range for the score. Lower is better for every metric except the score.

Two things stand out.

1. A demo is not a store. The demo loads 41 requests and 0.3 MB. The two real stores load 258 and 440 requests, and 7 to 18 MB: analytics, ad pixels, reviews, chat, personalisation, and on Store A's homepage a 7.8 MB video. Most of that doesn't come from the theme, and it doesn't disappear when you change the frontend.

2. Headless alone didn't make the real store faster, in my tests. Store A, on the same architecture as the demo, averaged 39. Store B, on a theme, averaged 43. The difference is smaller than their run-to-run variation. With two stores I can't say what is true everywhere, but here, what decided speed was what each page loaded, not how its frontend was built.

Going headless also adds work a demo doesn't show: a frontend to host and maintain, analytics and tracking to rebuild and verify, and apps that plug into the theme to reconnect or replace. None of this makes headless wrong. For some brands it is the right call. It means a demo score is not evidence for your store.

One test is not a measurement

Look at the ranges. Under identical conditions:

• The demo scored anywhere from 72 to 87.
• Store A's Largest Contentful Paint went from 2.6 to 5.0 seconds.
• I ran this website through the same setup six times as well. It scored from 71 to 97, and its Total Blocking Time went from 58 ms to 675 ms between two runs a few minutes apart.

If you test once, you can "prove" almost anything. One run of my site (71) scored worse than one run of the demo (87). Run each six times and you can see that their scores overlap, so neither has shown it is faster on score.

Not every number varies the same way. Request counts and page weight barely moved (under 5% variation). Most timings moved by 10% to 35%. Total Blocking Time moved the most, because it depends on what the CPU happens to be doing when scripts arrive. That tells you where to be careful.

How to test properly:

1. Fix the conditions. Same tool, same location, same device, same connection. GTmetrix lets you set all four. PageSpeed Insights doesn't let you choose, so repeat it more.
2. Run each page at least five times, interleaving the pages you compare.
3. Report the average and the standard deviation, not the best run. The average tells you where a page usually lands. The standard deviation tells you how much to trust a single result.
4. Only call it a difference if it is bigger than the noise. If two pages' ranges overlap, you haven't shown that one is faster.
5. Check real-user data too. PageSpeed shows what Chrome users actually experienced when a site has enough traffic. Lab tests tell you why. Field data tells you whether it matters.

Five checks before believing any speed claim

1. Test the real page, not the screenshot, and repeat it, as above.
2. Count what it loads. In Chrome DevTools, open the Network tab and reload. Look at requests, total size and how many domains appear.
3. Look for the tags. Search the page source for what a real business needs: analytics, ad pixels, reviews, chat, A/B testing. If they're missing, the score measures a page customers will never get.
4. Compare like with like. A demo against a store with hundreds of requests is not a test of the architecture.
5. Ask what was left out. A headline score with no conditions, no repeat runs and no real-user data is a starting point, not evidence.

Speed tricks that work, and what they cost

I applied these on this website. Some remove work. One mostly delays it, and that difference matters.

The second row deserves honesty. My tags still load, just later. GTmetrix waits long enough to see them, which is why its "fully loaded" time for my site was about 10 seconds while the main content appeared in about 1. It is also the most likely reason my Total Blocking Time jumped in some runs: the tags arrived while the test was still measuring. A test that doesn't wait, or a visitor who leaves in under 4 seconds, never loads them at all.

For a personal site, that is an acceptable trade. For a store, it usually isn't. If an ad pixel doesn't fire, the ad platform can't attribute the sale and the campaign looks worse than it is. A/B testing tools usually need to run before the page appears, or visitors see the original version flash before the variant. Delaying them to win a score makes a store look faster and makes every decision based on its data less accurate. Sometimes slower and accurate is the better business decision.

No magic pill: give someone the job

The pattern I see is companies waiting for the one change that fixes speed at once: a new platform, a new theme, a plugin. In my tests, two real stores on different architectures landed in about the same place, because both carry the same kind of weight. What works is slower and more reliable: someone who owns the problem, measures properly, changes one thing and measures again.

If you have a tech team, make one person the expert in performance, and give them AI to multiply their testing:

• Run more tests, faster. An AI assistant can start repeated runs, collect the results and compute averages and standard deviations. The 24 runs in this article were run and summarised that way.
• Read the reports for you. Each report lists the scripts, images and requests that cost the most. An assistant can compare them across runs and pages.
• Check claims before the team acts on them. An hour of proper testing beats a month spent on the wrong migration.
• Keep the judgment human. AI is fast at measuring and can be confidently wrong. Deciding which trade-off is right for the business stays with the person.

The rules change too: the metrics, how search engines rank pages, and how AI assistants pick which sources to cite. I wrote about how to keep track of those changes with AI in Best practices expire.

A specialist who tests a lot will beat a magic pill every time, because they will know what the score is measuring, and how much it moves on its own.