site-architecturelisted

# Site Architecture Skill For technical website structure that enables discovery by search engines AND AI crawlers (GPTBot, ClaudeBot, PerplexityBot). --- ## Philosophy **Content is king. Architecture is the kingdom.** Great content buried in poor architecture won't be discovered. This skill covers the technical foundation that makes your content findable by: - Google, Bing (traditional search) - GPTBot (ChatGPT), ClaudeBot, PerplexityBot (AI assistants) - Social platforms (Open Graph, Twitter Cards) --- ## robots.txt ### Basic Template ```txt # robots.txt # Allow all crawlers by default User-agent: * Allow: / Disallow: /api/ Disallow: /admin/ Disallow: /private/ Disallow: /_next/ Disallow: /cdn-cgi/ # Sitemap location Sitemap: https://yoursite.com/sitemap.xml # Crawl delay (optional - be careful, not all bots respect this) # Crawl-delay: 1 ``` ### AI Bot Configuration ```txt # robots.txt with AI bot rules # === SEARCH ENGINES === User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / # === AI ASSISTANTS (Allow for discovery) === User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Claude-Web Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Amazonbot Allow: / User-agent: anthropic-ai Allow: / User-agent: Google-Extended Allow: / # === BLOCK AI TRAINING (Optional - block training, allow chat) === # Uncomment these if you want to be cited but not used for training # User-agent: CCBot # Dis