Skip to main content

AI bot visits

Count which AI bots read your site and which pages they read, through a GitHub pull request, Cloudflare or a snippet. What Knownwise keeps, how visits are verified, and the limits.

Last checked against the product on October 17, 2026

These docs are in English.

AI assistants send bots to read web pages. Some fetch pages to answer a question someone just asked, some build a search index, and some collect pages to train models. AI bot visits counts those bots on your site, page by page, so you can see which pages they read and whether they get through.

The numbers appear on the AI bots page of each brand and in a tile on the brand overview.

Three ways to set it up

Each brand needs one of these. You set them up in Settings, under Integrations, in the AI bot visits for section for that brand.

  • GitHub pull request. Knownwise opens a pull request that adds one small file to your Next.js site. You review it and merge it. Best when your site is a Next.js app in a GitHub repository.
  • Cloudflare. You paste a read-only Cloudflare API token. Knownwise reads Cloudflare's analytics once a day and changes nothing on your site. Best when your site already runs behind Cloudflare, and the only option for a static site.
  • Snippet. You copy a short piece of code for Next.js, Astro, Express or any server that can send an HTTP request, and deploy it yourself.

Install through GitHub or the snippet and Connect Cloudflare walk through each one.

Which one to pick

You don't have to work it out. Knownwise looks at your site's last site check (where it is hosted and what it is built with) and your connections, then suggests one route with a Suggested badge. The suggestion shows first in Integrations, on the AI bots page and on the overview tile.

  • A Next.js site, or a site hosted on Vercel or Netlify that is not built with Astro, WordPress, Webflow or Shopify, gets the GitHub route. If GitHub is not connected yet, the action is Connect GitHub. If it is connected but no repository is linked to the brand, it is Pick the repository. With a repository linked, it is Add tracking.
  • A site behind Cloudflare gets Connect Cloudflare.
  • Anything else gets Set up the snippet.
  • While a route is being set up, Knownwise suggests nothing new. "Being set up" ends: a pull request that is closed or never made, a test that gives up, or a snippet key that sends nothing for 7 days hands the suggestion back to the route that fits.
  • Once tracking receives visits, the suggestion moves on to Google Analytics and Search Console if they are not connected.

You can use more than one route for the same brand. Each visit is still counted once: when your site reports visits, those are the numbers, and Cloudflare adds only the refusals it saw. The numbers say which route they came from.

What Knownwise keeps

We only see AI bots, never your visitors.

  • The GitHub file and the snippet check each request's user agent on your server. A request from a person, or from any bot that is not on Knownwise's list of AI bots, sends nothing.
  • For an AI bot request, the code sends the time, your host name, the page path, the user agent and, when your host sets a trusted header, the bot's IP address. It never sends the query string, cookies or the page itself.
  • The server checks the user agent again and drops anything that is not an AI bot.
  • Knownwise's own site checks visit your home page as GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot to test your firewall. Those requests carry a link to the probe page and are never counted as visits.
  • It uses the IP address once, to check whether the visit really came from the bot's company, and then discards it.
  • Knownwise keeps one count per day, per bot and per page. There is no raw request log.

The site key in the code can only send visits to Knownwise. It cannot read any of your data.

Cloudflare sends counts per day, bot, page and status code. It never sends visitor data.

Verified visits

Anyone can put "GPTBot" in a user agent. So Knownwise checks the IP address of each visit against the addresses the bot's company publishes. Each visit falls into one of four groups:

  • Verified. The IP is inside the company's published addresses. This is the real bot.
  • Unverified. The company publishes no address list, so there is nothing to check against.
  • Spoofed. The company publishes addresses and this IP is not one of them. Something is pretending to be that bot.
  • Unknown. No IP arrived, or the IP belongs to a CDN in front of your site rather than to the bot.

The AI bots page shows the verified share, as the Verified column and KPI. Every other visit counts as a visit but not as verified, so a low share means some visits may not come from the bot they claim to be.

Whether visits can be verified depends on where your site runs. The Next.js code only trusts the IP header that Vercel or Cloudflare sets, because anyone who reaches your server directly can forge the others. On any other host the code sends no IP, and every visit is unknown.

Cloudflare's numbers are never marked verified. With only Cloudflare connected, Verified shows 0%.

Where each number comes from

Every group of numbers carries a label:

  • Reported by your site. Counted by the GitHub file or the snippet, one request at a time.
  • Estimated from Cloudflare. Cloudflare's analytics, which are built from a sample of requests.
  • Reported by your site, with refusals from Cloudflare. Your site's own counts; Cloudflare's numbers feed only the Refused column, so nothing is counted twice.

Plans

AI bot visits are on paid plans, through all three routes. If an organization moves to Free, Knownwise stops counting: the site's code still gets an answer but its visits are not stored, the Cloudflare sync stops, and the weekly report drops its AI bots line. On Free, the AI bots page shows an example layout, not your data, and the Settings cards say the feature is on paid plans. The GitHub route opens its file as a pull request on every paid plan, Starter included. It is not a fix: it needs no plan with fixes and never uses one of the month's fixes. See Plans and billing.

The firewall check runs in every site check, on every plan, Free included. See The firewall warnings.

Limits

  • Counts from your site are held in memory and written every 20 seconds. If Knownwise's server restarts at the wrong moment, up to about 20 seconds of counts can be lost. Knownwise also writes them when it shuts down for a deploy, but it does not delay the shutdown for that, so a deploy can lose up to the same 20 seconds.
  • A successful Test it proves the route works. It is not a visit: the AI bots page, the overview tile, the weekly report and the agent wait for the first real AI bot.
  • Cloudflare's numbers are sampled estimates. On very busy days Cloudflare's row limit can cut them short, and the Cloudflare card then says those counts are a floor.
  • A request that your firewall or bot protection blocks never reaches your site's code, so the GitHub file and the snippet never see it. Cloudflare does see the requests its own firewall blocks.
  • Knownwise counts visits to the brand's own host only, plus www. when the brand is on the bare domain.
  • Static files such as images, scripts and fonts are skipped. robots.txt, sitemap.xml and llms.txt are counted.
  • One brand can count up to 200,000 bot visits a day.