Understanding Google Search Console's Crawl State: How Crawl Requests Change with Discovery

-

 In Google Search Console, the Crawl state is a crucial aspect that helps website owners understand how Google's web crawler interacts with their site. One significant factor influencing this state is the percentage of Crawl requests, which is further divided into two categories: "Refresh" and "Discovery." This article aims to shed light on how Google increases or decreases the percentage of these crawl requests based on website updates and content changes.

Accessing the Crawl State Report:
    To begin, log in to your Google Search Console account and navigate to the Crawl state report. This report provides valuable insights into how Google's web crawler interacts with your website.

Understanding "By Purpose":
    Within the Crawl state report, you'll find a section labeled "By purpose," which breaks down the Crawl requests into two categories: "Refresh" and "Discovery."

Analyzing a Specific Example - Date: 3rd July 2023: a) Refresh - 89%, Discovery - 11%:


During this period, we made significant updates to various sections of our website, including the footer section, Executive leadership page, Home page background video, and an Australian indigenous section. As a result, the percentage of Crawl requests classified as "Discovery" increased to 11%.

Another Case - Date: January 2023: a) Refresh - 96%, Discovery - 4% :
    

During this period, no substantial updates were made to our website. As a consequence, the percentage of Crawl requests classified as "Discovery" remained low, at only 4%.

The Crawl state in Google Search Console provides invaluable insights into how Google's web crawler interacts with our website. By analyzing the percentage of Crawl requests, specifically "Refresh" and "Discovery," we can better understand how our content changes impact the crawler's behavior. When updates are made to the website, such as adding new sections or modifying existing content, the percentage of "Crawl requests: Discovery" tends to increase, indicating that Google is actively exploring and indexing these changes. On the other hand, periods with fewer updates result in a lower percentage of "Crawl requests: Discovery." Understanding these patterns can help us optimize our website and ensure that Google effectively crawls and indexes our most relevant content.

Comments

Post a Comment

Popular posts from this blog

How I Achieved First Page First Position for High Search Volume Keyword on Google SERP

-
Image
In this post, I would like to share my experience of bringing one of our clients to the first page first position on Google SERP for a high search volume keyword. The client, Pothys, is a big brand in the textile industry with a strong reputation in offline marketing. However, their website was not optimized for basic SEO practices, which I discovered during my initial audit. Pothys was facing a common challenge among businesses - ranking high on Google for a high search volume keyword, "Silk Sarees." When I first analyzed the keyword position, the website was ranking at 7th position on the first page of Google SERP. But Pothys was determined to reach the first position. The search volume of the keyword "Silk Sarees" is (49500/month). I identified several issues with the website that were hindering its search engine ranking. These issues included: Keyword cannibalization : The same keyword "Silk Sarees" was targeted on multiple pages on the site, which was...
Read more

Crawled - Currently Not Indexed

-
Image
 What Google's official document says about "Crawled - currently not indexed": "The page was crawled by Google but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling." Source: https://support.google.com/webmasters/answer/7440203?hl=en Case1: I don't want to index these pages (Category, Tag, Dev & Feed): these pages are already marked as "no-index" in the Meta robots tag.  https://scgroup.global/blog/category/ceo-sleepout/ https://scgroup.global/blog/page/9/ https://scgroup.global/blog/category/technology/electronic-security/ https://scgroup.global/blog/annual-ceremony-awards/a2-1024x768/ ttps://scgroup.global/blog/tag/outstanding-guarding-company/ https://scgroup.global/blog/category/events/ https://scgroup.global/blog/hire-commercial-cleaning-service-provider/feed/ https://scgroup.global/blog/mental-health-in-security-industry/feed/ https://scgroup.global/dev2022/about/advisory-board/ https://sc...
Read more

Crawled - Currently not indexed - My Experience Troubleshooting the Issue

-
Image
 As a SEO Analyst, one of the challenges I faced was encountering a "Crawled - Currently not indexed" status on Google for our old blog post - https://scgroup.global/blog/safety-tip-green-cleaning-cleaner-future/ (Continuity of case 2 ) that I worked on. After some analysis and troubleshooting, I was able to fix the issue and get the post indexed. First , I analyzed the content, meta tags, internal, and external links for the blog post. I used the "Copyscape" tool (https://www.copyscape.com/) to check for any plagiarism issues with the content. I then edited and updated the meta tags to make sure they were properly optimized for the post.  The title and meta description for the post were updated to : Title: Safety Tip: Green Cleaning for a Cleaner Future | Cleaning tips Meta Description: When you choose green cleaning products and practices, your family is safer and healthier. Find out more about the advantages of greener cleaning. The future: green cleaning, clean ...
Read more