Your traffic dips, new posts aren't showing up in Google, and Search Console is giving you vague answers. Before you rewrite content or buy an SEO audit, answer the more basic question: is Googlebot actually visiting your pages at all?
This became urgent on WP AI Success Hub calls when Google had an indexing outage and one member found 82 of his pages had dropped out of the index. Here's how we diagnose it.
Start with the site: search
Search site:yourdomain.com in Google. It's not a perfect count, but a big gap between how many pages you have and how many Google shows is your first signal. Check a few specific important URLs with site:yourdomain.com/that-page/.
Use Search Console properly
- URL Inspection on a specific page tells you when Google last crawled it and whether it's indexed
- Pages report (Indexing → Pages) shows the indexed vs excluded split and the reasons — watch for sudden jumps in "Crawled – currently not indexed"
- Crawl stats (Settings → Crawl stats) shows Googlebot's actual visit volume over time — a cliff here means a real problem
Know that "discovered" isn't "indexed"
Google finding your URL, crawling it, and indexing it are three separate steps, and pages can stall between any of them. That distinction matters during Google-side outages too — sometimes the problem is genuinely not your site. Before panicking (or trying risky "force reindex" hacks — we advise against them), check whether others are reporting the same thing.
Beware fake Googlebots in your logs
If you look at server logs or a bot-tracking plugin, know that plenty of crawlers claim to be Googlebot. Real Googlebot visits come from Google's published IP ranges and can be verified — good tracking tools do this check for you. Don't celebrate crawl activity that's actually scrapers.
The basics that fix most crawl problems
- XML sitemap submitted in Search Console (Yoast generates one automatically)
- No accidental noindex on important pages (a depressingly common find)
- Internal links to every page you care about — orphan pages get crawled last
- Reasonable site speed — Google crawls slow sites less
Get it diagnosed on screen
Indexing problems are miserable to debug alone. On WP AI Success Hub live calls we open members' Search Console together and work out whether it's your site, your content, or just Google having a week. Join for $27/month.