When your CDN silently strips the User-Agent

A request that arrives with no User-Agent is a claim about your infrastructure, not about the visitor. We learned this by blocking nobody and counting carefully.

An empty User-Agent is not proof of a bot. It is proof that the header did not survive the trip. Any layer that reconstructs the upstream request — a Cloudflare Worker, a middleware, a rewrite rule, a proxy — can drop it, and nothing in the response tells you. In June 2026 we measured 6,661 genuine sessions arriving with no User-Agent on customer sites, overwhelmingly on residential and mobile ISPs. The cause was a Worker that was not always forwarding the header.

How we found it

The prompt was a proposal to block traffic with no User-Agent. That is a reasonable-sounding rule — scanners frequently send nothing — so before agreeing we looked at who it would actually hit. The answer, measured against customer hosting logs, was that it would have hit real people: the empty-UA rows sat on consumer ISPs across several countries, not on datacenter ranges, and they behaved like ordinary visitors rather than like scanners.

The lesson generalises past this one header. Before adding a block rule, query what it would have caught. A rule that seems obviously correct against an imagined attacker can be a rule against your own customers, and the only way to know is to look at the rows.

Before and after the fix

MeasureJune 2026October 2026
Allowed sessions with an empty User-Agent6,6610
Their ISPsResidential and mobile, multiple countries—
CauseCloudflare Worker not forwarding useragentFixed

Worth being explicit about the second column: the symptom no longer occurs on our platform. We are publishing the June figure as a dated case study, not as a current statistic — the fix was to make the Worker forward the header reliably, which is the only durable answer. Once an empty User-Agent becomes rare, it becomes meaningful again. Until then it is noise wearing the costume of a signal.

Check your own stack in five minutes

The test is to send a User-Agent you can recognise and see whether it survives to your origin. Use something no real client would send:

curl -A "TestUA-ZB-CHECK-9f3a" https://yoursite.example/ # then grep your origin log for that literal string. # # present -> the header survives every layer # absent, or "" -> something in the path is dropping it # (Worker, proxy, middleware, rewrite)

Run it through every route a visitor can take, not just the one you think is the main path. The layer that drops headers is rarely the one you would nominate, and on our own platform it was a component that only failed intermittently — which is why it went unnoticed for so long.

If you integrate a detection API, run the same test on the value you forward, not just on what your origin receives. A header that survives to your application can still be lost on the way out, and the effect is identical: every one of your real users arrives looking automated.

This is why ZeroBot treats an empty User-Agent as a weak signal to combine with others rather than a verdict, and why the API returns the individual signals alongside the score — so you can see that a verdict rests on a header that may be missing for reasons that have nothing to do with the visitor.

Frequently asked questions

Can a CDN or proxy strip the User-Agent?

Yes, and it does so silently. Any component that reconstructs the upstream request — a Worker, a middleware layer, a rewrite rule, a proxy with its own header handling — can fail to forward a header, and nothing in the response tells you. You find out by looking at what your own logs recorded.

How do I check whether something in my stack drops the User-Agent?

Log an intentionally unusual User-Agent through your real path and fetch it through every layer. Send <code>TestUA-ZB-CHECK-9f3a</code> and look for that literal string in your origin logs. If it is missing or replaced with something empty, you have found the layer that drops it.

Why does an empty User-Agent cause false positives?

Because "no User-Agent" is a common heuristic for automation, so any rule that blocks or heavily penalises it will also block the real users whose requests arrived with the header stripped. The signal is about your infrastructure, not about the visitor.

Is an empty User-Agent ever a genuine bot signal?

Sometimes — scanners frequently send nothing at all. The problem is that it is not exclusively a bot signal, and a rule cannot tell the difference between a scanner and a real visitor whose header was dropped upstream. Treat it as a weak signal to combine with others, never as a verdict.

Check the signals, not just the score.

Every verdict comes back with the reasons behind it. $50/mo flat, free tier.

Get started free →

Related

Does bot detection actually detect? Anti-bot API Residential proxies and IP reputation What bot protection costs