The Knownwise probe
If your logs show a user agent with +https://knownwise.com/ai-bots/probe or KnownwiseProbe, the request came from Knownwise. Here is what it does, why, and how to stop it.
The firewall check
When someone runs an AI-readiness audit, we request the page they entered five times in a row: once as an ordinary desktop browser, then once each as GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot. The bot requests use each company's published user agent with our marker added, so you can tell them apart from the real bots.
Why: if the browser gets the page and a bot gets a block or a challenge, your firewall is probably turning the real bots away too. robots.txt can say yes while a firewall says no, and that is hard to see from the inside.
The install check
Only for sites whose owner set up AI bot tracking in Knownwise. We request the homepage once as KnownwiseProbe, with a one-time code, to confirm the tracking install works. If the code does not arrive, we try again a few times over the next day, at most seven requests, then stop.
KnownwiseProbe/1.0 (+https://knownwise.com/ai-bots/probe)
Small, read-only, and only when asked
- Only the page entered for the check, usually the homepage, with a plain GET.
- Five requests per audit, one after another, following at most three redirects each.
- No forms, no logins, no JavaScript, and no other pages followed.
- Each request stops after 8 seconds and reads at most 64 KB.
- Cookies your site sets are thrown away.
How to block or allow it
To block it
Add a firewall rule for user agents that contain knownwise.com/ai-bots/probe or KnownwiseProbe. robots.txt does not cover the firewall check: it fetches one page that a person asked about, the way a browser would.
If you block only our marker, the audit will report that your firewall turns AI bots away, even if the real bots get through. That result then says nothing about the real bots.
To allow it
We suggest no special rule. The check is only useful when your firewall treats it like the real bots, so an allow rule for our marker would hide the problem it is there to find.