Google has added a new crawler to its list of Google Crawlers and user agents, this one is named GoogleOther. It is described as a "generic crawler that may be used by various product teams for ...

Google has launched a major revamp of its Crawler documentation, shrinking the main overview page and splitting content into three new, more focused pages. Although the changelog downplays the changes ...

Mihai Aperghis asked Google's John Mueller about the Google Search Console crawl stats report asking if it includes data on just HTML pages or also images, PDFs, etc. John said not only does it ...

Google has fixed a typo in their crawler documentation that inadvertently misidentified one of their crawlers. In general, this is a minor issue but it’s a major issue for SEOs and publishers who ...