On Tuesday I did something I tell clients to do and had not done carefully enough on my own site. I opened Google Search Console and actually read it, rather than glancing at the graph.
My pricing page had never been crawled. Not ranked badly, not buried on page four. Google had never once fetched it. The same was true of my process page and two blog posts.
That is a page explaining what I charge, on a site whose entire job is to get people to enquire, and it did not exist as far as Google was concerned.
Ranking badly and not being there are different problems
Most small business owners I speak to assume their site is on Google and simply not very high up. That is a reasonable assumption and it is often wrong.
Being indexed and ranking well are two separate things. Indexing is whether Google has the page at all. Ranking is where it puts that page once it does. You can work on ranking for months and get nowhere if the page was never indexed, because there is nothing to rank.
It is the difference between being last in the queue and not being in the building.
How to check, in about two minutes
The rough version takes ten seconds. Go to Google and search for this, replacing the domain with your own:
- site:yourbusiness.co.za
That shows you roughly what Google holds for your site. If you have twelve pages and it shows four, you have found something. If it shows nothing at all, you have found something quite serious.
The proper version is Google Search Console, which is free and takes a few minutes to set up. Open Indexing, then Pages. It splits your site into indexed and not indexed, and tells you the reason for every page in the second group.
Some of those reasons are completely fine. A page that redirects elsewhere, or one you have deliberately hidden, will sit in that list and should. The two worth caring about are these:
- Discovered, currently not indexed. Google knows the page exists and has not bothered going to look. This is the common one on newer sites.
- Crawled, currently not indexed. Google went, read it, and decided not to keep it. That one usually means the page is thin or duplicates something else.
Why it happens to small sites in particular
Google does not crawl every site equally. It allocates attention based on how important it thinks a site is, and a new site that almost nothing links to sits near the bottom of that list.
So it visits, takes what it can be bothered with, and leaves. Pages linked from a lot of places get picked up. Pages sitting quietly in a corner do not.
In my case there was a second cause and it is a common one. Your sitemap is the file that tells Google what pages exist. Mine was submitted correctly and Google had read it exactly once, three weeks earlier. Everything published since was listed in a file nobody was looking at.
Worth knowing: submitting a sitemap once is not the same as Google reading it regularly. On an established site it will come back on its own. On a new one it often will not, and you can go months assuming you are covered.
What to actually do about it
Three things, in this order, and none of them cost anything.
- Request indexing on the missing pages. In Search Console, paste the URL into the inspection bar at the top and click Request Indexing. It pushes that page up the queue and usually gets it crawled within hours rather than weeks. Do your money pages first, not your blog.
- Resubmit your sitemap whenever you publish something. It takes two clicks and it forces Google to re-read the file instead of trusting it to come back.
- Link to your own pages from your own pages. Google partly finds things by following links. A page that nothing on your site links to is a page it has very little reason to visit. One of my own posts had no internal links pointing at it at all, which is the single best way to make something invisible without realising.
The part nobody enjoys hearing
Requesting indexing fixes the symptom. It does not fix the cause.
The reason Google was not bothering with my pages is that very little on the internet links to my site, so it has no particular reason to think the site matters. That is not solved by a button. It is solved slowly, by being linked to from real places, which is harder and less satisfying than publishing another article.
Which is the genuinely useful lesson in this. I had written eight blog posts while the page that tells people what I charge had never been looked at. More content was not the answer. Checking was.
Do this before you spend money on SEO
If someone is about to sell you a monthly search package, run the site: search first and open Search Console. If half your pages are not indexed, you do not have a ranking problem yet, and paying for ranking work would be solving the wrong thing in the wrong order.
It is a ten minute check. I nearly did not do it on my own site, and I do this for a living.
If you want someone to go through this properly on your site, the Process page explains how I work and the Pricing page covers what a build costs. If you are earlier than that and still weighing up whether a site of your own is worth it at all, this one covers the difference between having a site and being listed on someone else's.