Once in a while I receive a frantic email that someone’s website completely disappeared from the internet and is no longer showing up in search engines.
The problem is usually detected when a webmaster or business owner searches for a keyword that would certainly bring up his website if it was indexed in Google. This is not foolproof, however. The best way to know whether a website is indexed is simply to make a search for the URL of the site’s domain. For example, if I wanted to check if this website is indexed I would search for https://www.seoagencyllc.com. If the website doesn’t show up in the first few results on the SERP, and certainly if it is not on the first page of Google at at all, then I would know with certainty that my website is not in Google‘s index.
Occasionally, a situation will occur where most of the pages of a site are indexed and properly appearing in Google search. However, the webmaster will contact me and say, “Please help, my Homepage is not showing up in Google!”
This can also be verified just as before, by searching for the URL‘s of all the pages including the Homepage and seeing which pages show up in Google.
It is obviously very distressing to discover that your Homepage, or any page, for that matter is not showing up in Google search. All of the work that went into designing your website, such as purchasing a domain, setting up hosting, and writing content etc., seems to have been almost for naught.
Sure, the business owner, or webmaster, still has an online address to showcase his services or products, but that is small comfort for someone who was counting on driving traffic to his business via search. He only has the option to send people directly to his website, but he will not really be able to get new customers through his website. And that hurts.
In these situations I like to do what I can to help because I understand the pain that the business owner is expressing when he sends me the email with the subject line: My Homepage Is Not Showing Up In Google. I know right away that this person feels great frustration and is desperate for a solution.
Therefore, as soon as I can, I begin to research and dissect the website from a technical standpoint, examining many possible causes for the disappearance of the website.
Let me tell you what I discovered is probably one of the most common reasons for the website’s disappearance.
It has to do with Robots Tags. No, not the kind of Robot that you are used to. I’m talking about the search engine Robots, also called spiders or crawlers. Known as “Bots” for short, these programs crawl the internet, going from one page to another and one site to another, and record the information that they discover. Google‘s web crawler is called the “Googlebot.”
It is these Robots that enable your website to be indexed. They report back, so to say, to the search engine that sent them, and inform them of the information they gathered when they were crawling the web.
The thing about search engine Robots is that they can be controlled, or directed by your website. You can insert tags, called “meta tags” into the code of your webpage that directs the crawler not to index your site.
Yep, you decide whether a page should be indexed or not.
This tag is a directive to search engine Robots, telling them not to include the site in their index.
When this tag is applied to your site’s Homepage it will not get indexed. The same thing with any other page of your site.
The good news is that it is very easy to fix this. All you have to do is simply remove the the tags and the next time your site is crawled by the search engine spiders it will get indexed. How to remove the tags depends on what platform or CMS (Content Management System) your site was built with. It also depends which SEO plugin you are using in WordPress, if that is what you used to created your site. Poke around a bit in the settings and you will find the place where you can have the “noindex” tag removed.
Once you have successfully removed the “noindex” tag from your site, there is a little trick you can do to speed up its indexing. You can submit your website to Google and they will send out their crawlers to spider your website and add it to their index.
The easiest way to do this is simply to google, “submit URL to Google”. This box will come up:
All you need to do then is to just enter your URL into the box, press submit, and wait for your page to get indexed.