What are "Other agent type" Googlebot types in GSC crawl stats?

If you’re not running Google Ads or producing a large volume of new content, chances are it’s not AdsBot or something new being discovered. Unless you're experiencing significant issues with your site, it’s usually nothing to stress over and could simply be background activity from Google.

I honestly think sometimes Google's little robots are on some sort of secret mission, checking things out or finding new ways to explore the web. They don’t always let us know what they’re up to!

Or you could see it as a cool job by Google’s bots: Maybe they’re searching for something super specific in your little corner of the internet that only those curious bots know about.

So really, there’s probably no reason to freak out. It simply means that Google sent some of its less common, specialised bots to see what’s going on in your space. They're just doing their thing to help people discover your corner of the internet!

Unless you're experiencing significant issues with your site

Oh, but I am experiencing significant issues with my site. I also just noticed that within 72 hours of the spike, 80% of our URLs have been deindexed by Google. This is the first time this has ever happened in our 23 years.

Unless you're experiencing significant issues with your site

Oh, but I am experiencing significant issues with my site. I also just noticed that within 72 hours of the spike, 80% of our URLs have been deindexed by Google. This is the first time this has ever happened in our 23 years.

Oh wow, Dani, that 80% deindexing changes things – that's a massive issue and sounds connected to that "Other agent type" spike. For a site of your age, my first thought is usually a manual action or a security issue from Google. Have you had a chance to check your GSC under Manual actions and Security issues yet? That's usually where Google will tell you directly if something severe happened. Please keep us posted; this needs a thorough review.

No manual actions or security issues.

No manual actions or security issues.

Okay, Dani, that's genuinely good news about no manual actions or security issues, at least that rules out the scariest stuff! But it makes the 80% deindexing even more puzzling.

Since it's not a penalty, my mind immediately jumps to some kind of technical configuration problem that might have happened on your end, possibly right around that Other agent type crawl spike.

You should thoroughly review your robots.txt file and check for any noindex tags that may have been added. Additionally, could you please check your server logs for any sudden increase in 404 errors or server errors? That rapid deindexing suggests that Google is being blocked or instructed not to index a significant portion of your site.

Yesterday I ran a Screaming Frog SEO spider crawl of my site as a Googlebot user agent to see if there was some reason that Google was being blocked, and nothing unusual showed up.

Google crawl stats in search console don’t indicate anything out of the ordinary such as 4xx or 5xx errors.

I posted yesterday in the Google Webmaster Central forums and encountered someone else who posted that the exact same thing happened to them on the same exact day.

Yesterday I ran a Screaming Frog SEO spider crawl of my site as a Googlebot user agent to see if there was some reason that Google was being blocked, and nothing unusual showed up.

Google crawl stats in search console don’t indicate anything out of the ordinary such as 4xx or 5xx errors.

I posted yesterday in the Google Webmaster Central forums and encountered someone else who posted that the exact same thing happened to them on the same exact day.

IMO, If other people are noticing this same deindexing right after a spike in Other agent type, it seems more like it could be a Google bug that hasn’t been reported yet or a new, really strict automated filter that’s hitting certain types of sites or setups rather than something wrong with your site itself.

I posted a question over in Google's webmaster forums for some more details. As a status update, the mass deindexing seems to be happening on a large enough scale, across enough sites, that Barry Schwartz wrote an article about it last week.