← ClaudeAtlas

ai-logslisted

Count real AI and search crawler visits in server access logs and separate verified bots from fakes using each vendor's published IP ranges (OpenAI, Google, Bing, Perplexity). Also counts visitors referred by ChatGPT, Perplexity, Copilot, Gemini and Claude. Use when the user asks 'is ChatGPT/GPTBot actually crawling my site', 'which AI bots visit us', 'are these bots real', 'how much traffic comes from ChatGPT', or shares an access.log / nginx / Apache / Cloudflare log export.
sitetiller/ai-visibility-skills · ★ 0 · AI & Automation · score 75
Install: claude install-skill sitetiller/ai-visibility-skills
# AI crawlers in your logs robots.txt says who may visit. Logs say who did. Anyone can put "GPTBot" in a user agent, so a hit only counts as OpenAI's when the IP is inside OpenAI's published ranges. This skill reports both. ## Inputs - One or more access logs in common or combined log format (nginx and Apache defaults). Gzip: decompress first. - Other formats (Cloudflare Logpush JSON, CDN CSV): convert to combined format, or ask for the columns ip, time, request, status, referer, user agent and convert with a short script. ## Steps 1. Run the analysis (downloads the vendors' IP lists once per run): ```bash npx sitetiller logs access.log access.log.1 --json > ai-logs.json ``` Offline: `--no-verify` counts by user agent only and marks every hit unverifiable. 2. Read per bot: `hits`, `verified`, `spoofed`, `unverifiable`, `ips`, `status` mix and top `paths`. 3. Interpret: | Finding | Meaning | Action | |---|---|---| | `verified` hits on search bots (OAI-SearchBot, PerplexityBot, Bingbot, Googlebot) | The engines can read the site | Check that top paths are the pages you want cited | | Search bot hits mostly `4xx/5xx` or `403` | Bot reaches the server but is refused | Check WAF/CDN rules and robots.txt (`ai-crawlers` skill) | | `spoofed` > 0 | Scrapers using a vendor's name | Safe to rate-limit or block those IPs; it does not affect the real bot | | `unverifiable` | Owner publishes no IP list (Anthropic, Apple, Amazon, Meta, others) | Treat counts as an upper boun