seo-technical

Featured

Audit technical SEO across crawlability, indexability, security, URLs, mobile, Core Web Vitals, structured data, JavaScript rendering, and related platform signals like robots.txt and AI crawler access.

AI & Automation 49,938 stars 7238 forks Updated today MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Technical SEO Audit ## When to Use - Use when the user wants a technical SEO review focused on crawlability, indexability, performance, or rendering. - Use when auditing robots.txt, canonicalization, JavaScript SEO, Core Web Vitals, or AI crawler access. - Use when the task is infrastructure- and implementation-oriented rather than content-focused. ## Categories ### 1. Crawlability - robots.txt: exists, valid, not blocking important resources - XML sitemap: exists, referenced in robots.txt, valid format - Noindex tags: intentional vs accidental - Crawl depth: important pages within 3 clicks of homepage - JavaScript rendering: check if critical content requires JS execution - Crawl budget: for large sites (>10k pages), efficiency matters #### AI Crawler Management As of 2025-2026, AI companies actively crawl the web to train models and power AI search. Managing these crawlers via robots.txt is a critical technical SEO consideration. **Known AI crawlers:** | Crawler | Company | robots.txt token | Purpose | |---------|---------|-----------------|---------| | GPTBot | OpenAI | `GPTBot` | Model training | | ChatGPT-User | OpenAI | `ChatGPT-User` | Real-time browsing | | ClaudeBot | Anthropic | `ClaudeBot` | Model training | | PerplexityBot | Perplexity | `PerplexityBot` | Search index + training | | Bytespider | ByteDance | `Bytespider` | Model training | | Google-Extended | Google | `Google-Extended` | Gemini training (NOT search) | | CCBot | Common Crawl | `CCBot` | Op...

Details

Author
sickn33
Repository
sickn33/antigravity-awesome-skills
Created
10 months ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category