BensonBot: what it fetches and why
What BensonBot fetches, when, how often, its User-Agent, and how to ask us to stop. It never crawls.
BensonBot is the name Benson uses when it fetches a web page. If you've seen it in your logs, here's exactly what it was doing.
What it fetches#
BensonBot fetches a store's homepage when someone asks Benson to look at that store:
- a store owner adding their site in Benson's setup wizard, or
- a visitor previewing Benson on trybenson.com for a store address they typed.
A check fetches GET /, and on Shopify stores GET /cart.js to confirm the platform. It reads the page to recognise the platform, Google Tag Manager, a consent banner and any Content Security Policy, then stops.
The preview also shows the store's name and icon, so it fetches the icon the homepage names, or /favicon.ico if it names none or that one fails. That's at most two more requests, for images of up to 64KB. A check never makes more than four requests in all.
What it doesn't do#
- It never crawls. It doesn't follow links, index pages or come back on a schedule.
- It doesn't run your scripts, submit forms, or log in.
- It reads at most 2MB and follows at most 4 redirects, and only fetches public addresses (never private or internal networks).
User-Agent#
BensonBot/1.0 (+https://trybenson.com/bot)
How often#
Only when a person asks. Checks are rate-limited per visitor, and preview results are cached for a day, so repeated previews of the same store don't fetch it again.
Opting out#
Because BensonBot only fetches pages people ask about, there's nothing to schedule or throttle. If you'd still like us to stop checking your site, email [email protected] with your domain and we'll stop checks of it.
Stuck? Email [email protected].
