Our crawler
SiteStyleGuideBot
If you’ve seen SiteStyleGuideBot in your logs, someone added your website to SiteStyleGuide, which makes one-page style guides of public websites. Here’s what it does and how to stop it.
How it identifies itself
SiteStyleGuideBot/1.0 (+https://sitestyleguide.com/bot)
The same name is sent with every request it makes, including the screenshots it takes with a headless browser.
What it fetches
- Your
robots.txt, first. - Your homepage, once.
- The stylesheets, web manifest, favicons, logo and sharing image the homepage links to.
- Two screenshots of the homepage, at desktop and phone widths, which load the page as a browser would.
It never follows links to other pages, submits forms or signs in. A site is read again only when its guide is refreshed, which members can do once every 90 days. Captures across all sites are limited to a few every 15 minutes, so it never visits in bursts.
How to stop it
SiteStyleGuideBot follows robots.txt. To keep it off your site, add:
User-agent: SiteStyleGuideBot
Disallow: /
It also obeys a User-agent: * group that disallows your homepage. A site that blocks it this way can’t get a new guide.
To remove a guide that already exists, use the takedown form. Removal is free, and the site can’t be added again.
Contact
Questions or problems: hello@sitestyleguide.com.