What Crawl Budget Means for Google search bot
Crawl rate limit: The crawl rate can go up and down based on a couple of factors: UP: If the site responds really quickly for a while, the limit goes up, meaning more connections can be used to crawl. Down: If the site slows down or responds with server errors, the limit goes down and Googlebot crawls less. Crawl demand: The two factors that play a significant role in determining crawl demand are: Popularity: URLs that are more popular on the Internet tend to be crawled more often to keep them fresher in Google index. Staleness: our systems attempt to prevent URLs from becoming stale in the index. Factors affecting crawl budget: According to Google analysis, having many low-value-add URLs can negatively affect a site's crawling and indexing. Faceted navigation and session identifiers On-site duplicate content Soft error pages Hacked pages Infinite spaces and proxies Low quality and spam content Source: https://developers.google.com/search/blog/2017/01/what-crawl-budget-means-for-go...