The symptom
You inspect your page and everything is correct. Yet the URL Debugger, Google, or Facebook reports missing titles, different text, or tags you removed months ago. The server is serving different HTML to different clients.
Why it happens
- Server-side user-agent sniffing: your hosting or a plugin serves a stripped-down version to bots (often to "optimize for crawlers"). Google treats heavy divergence as cloaking.
- JS-rendered content: if meta tags or body content are injected by JavaScript, crawlers that do not execute JS see none of it.
- A/B testing or geo/device targeting that returns different markup depending on request headers.
- A CDN or caching layer serving stale HTML to bot requests.
Diagnose it — in order
- 1
Compare crawler views
In the URL Debugger, enable "Compare Googlebot & Facebook views". The table shows title, description, OG image, robots, and schema for your default view and each crawler. Amber cells are the mismatches.
- 2
Inspect the raw server response
The debugger shows the actual headers (content-type, x-robots-tag, cache-control) and the redirect chain — evidence of what the server really returned.
- 3
Test with curl
Run the page with a Googlebot user agent and again with a browser agent. Diff the two HTML outputs — that diff is exactly what crawlers see.
- 4
Disable JS rendering checks
If markup only exists after JS runs, the served HTML will not contain it. Check whether your tags are in the raw HTML or injected at runtime.
How to fix it
- 1
Remove user-agent divergence
Serve the same HTML to every client. If you optimize for bots, the content must match what real users see, or Google treats it as cloaking and may drop the page.
- 2
Move critical tags into server HTML
Title, description, canonical, OG tags, and JSON-LD should be in the initial HTML response, not added by client-side JavaScript.
- 3
Review CDN and cache config
Ensure the CDN is not serving a stale or cached variant to crawlers. Purge caches after changes and re-audit.
- 4
Re-audit until views match
Re-run the URL Debugger with the crawler comparison until all rows show identical values. That is the closest you can get to "what Google sees equals what I see".
Free tools that help
Still not sure what your page is sending?
Run your URL through the free URL Debugger — it shows the raw HTML evidence, redirects, headers, and what Googlebot and Facebook each receive.
Debug a URL free