How to get Google to crawl your site? There are a few ways to get Google to crawl your site, and index your posts and pages faster. In this article, I am sharing two strategies you can use right now. One is the most common method, and the other is a very unique, and not so well known strategy.
Before we get started, if you find the unique strategy helpful, be sure to check out my Blog Secrets 101 tutorial. It is filled with even more “think out of the box” website traffic and SEO strategies that will help your blog or website grow. Use promo code BLOG15 to get 15% off for a limited time.
Whether you are getting the dreaded “Discovered – currently not indexed” message in Google’s Search Console, or you simply want to get your blog posts and pages indexed faster (who doesn’t, right?), the below steps will get your site crawled much faster than just sitting and waiting weeks and months.
#1 – Common Method: Request Indexing in Google Search Console
If you have a self-hosted website, or blog through one of Google’s properties, and you are able to validate your site in Google Search Console, you can simply request indexing of your sitemap, or individual pages and blog posts. Here’s what you need to do:
12-24 hours after publishing your pages/ posts, access GSC and submit your URLs.
2. Select a property. If you have a new site and want to submit your main domain, select Domain. This will allow you to validate your site and submit the sitemap. The following steps are for individual URLs. If you have a specific URL you want crawled, select URL prefix.
3. Click on “URL Inspection” from the left sidebar menu.
4. From top search field, submit the URL for a specific blog post or page.
5. Google will run a check to see if there is data on the URL.
6. Click the REQUEST INDEXING link, if your URL is not found.
7. Check back with GSC in another 24 hours to see if the URL has been indexed.
You can also check if your pages are indexed without logging into GSC. Visit Google.com, type site:yourpageurl.com into the Google search box. If you only search for your homepage, it will show all indexed pages under the domain, but blog post URLs will only display for the specific page.
#2 – Unique Strategy: Increase Crawl Budget
What is Crawl Budget?
Crawl Budget is the number of pages Googlebot crawls and indexes on a website within a given timeframe. In short, if Google doesn’t index a page, it’s not going to rank for anything. Pages have to be crawled and indexed before they can rank. If your number of pages exceed your site’s crawl budget, you are going to have unindexed pages on your website.
For every website on the internet, Google has a fixed budget for how many pages their bots can and are willing to crawl. Googlebot can only spend so much time crawling and indexing sites.
Crawl budget optimization is the process of ensuring that the right pages of our websites end up in Google’s index and are ultimately shown to searchers. Your rankings are then determined on the backlink profile, search intent and content quality of your website and pages.
That being said, Google loves popular content, and sites with frequent posts. Therefore, it crawls popular sites and sites with frequent posts more often. So here’s what you need to do…
Crawl Budget Solution
This solution fixes “Discovered – currently not indexed”, and it gets your site crawled faster.
If your blog posts status in Google console show as “Discovered – currently not indexed”, there is a simple solution for new blogs and old blogs which haven’t been updated for a long time. You need to increase your Crawl Budget.
“Discovered – currently not indexed” in Google Search Console means Google is aware of your URL/ page/ post but has not crawled or indexed them yet.
This strategy gets Google to believe you are posting to your site multiple times a day, to get more frequent crawls.
Schedule 3 posts per day, 8 hours apart, and when each article goes live, resubmit your sitemap to Google. After a week or so, Google will start crawling a new article in about 1 hour, and index it in around 2 hours.