As just about every single Agents website is set up completely wrong in relation to optimisation for indexing your website in Google I am going to publish the basic. This information comes directly from Google itself. So pass it on to your web developer and tell them to get it right.
A. Submit your URL
1. The basics
Google is a fully automated search engine, which employs robots known as ‘spiders’ to crawl the web on a monthly basis and find sites for inclusion in the Google index. Since this process does not involve human editors, it is NOT necessary to submit your site to Google in order to be included in our index. In fact, the vast majority of sites listed are not manually submitted for inclusion.
Google does not accept payment for inclusion (known as “paid inclusion”) of sites in our index, nor for improving the rank of sites in our results. We do offer advertising opportunities adjacent to our results, which are always clearly labeled “Sponsored Links.” The method by which we find pages and rank them as search results is determined by the PageRank technology developed by our founders, Larry Page and Sergey Brin.
2. Submitting your site
We add thousands of new sites to our index each time we crawl the Web, but if you like, you may submit your URL as well. Submission is not necessary and does not guarantee inclusion in our index. Given the large number of sites submitting URLs, it’s likely your pages will be found in an automatic crawl before they make it into our index through the URL submission form. We DO NOT add all submitted URLs to our index, and cannot predict when or if they will appear.
Please visit our “Add URL” page to input your URLs. You can submit your site as often as you like, but multiple submissions will not improve the likelihood of your site being added or accelerate the process. We do not penalize sites for ‘over-submitting’. If you choose to submit your site, only the top-level domain is necessary, as the spiders will follow your internal links to all the rest of the pages.
The best way to ensure Google finds your site is for your page to be linked from lots of pages on other sites. Google’s robots jump from page to page on the Web via hyperlinks, so the more sites that link to you, the more likely it is that we’ll find you quickly.
B. I’ve submitted my site to Google and it’s still not listed. Why?
Google finds sites through a process known as “crawling” the web. This involves robot software that follows hyperlinks from site to site. Google currently looks at more than 8 billion URL’s during the crawl.
When a URL is submitted to Google, we look for it in our next crawl. If you’ve already submitted your URL, your site could easily appear in our new index, which will go up when the current crawl is completed. However, if no other site links to yours, it may be difficult for our crawler to find you. Conversely, if many sites link to your page, there is a good chance we will find you without your submitting your URL.
Occasionally, websites are not reachable when we try to crawl them because of network or hosting problems. When this happens, we retry multiple times, but if the site cannot be crawled, it will not be listed in our current index. If it was a transient problem, the site will likely show up in the next index, which will be completed in a few weeks.
If we have not picked up your site and it has been several months, then it is likely that our spiders are not able to find your site. If you increase the links pointing to the page, Google will likely find your site in the future.