site-crawlability

# SEO Technical: Crawlability Guides crawlability improvements: robots, X-Robots-Tag, site structure, and internal linking. **When invoking**: On **first use**, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On **subsequent use** or when the user asks to skip, go directly to the main output. ## Scope (Technical SEO) - **Redirect chains & loops**: Fix multi-hop redirects; point directly to final URL - **Broken links (4xx)**: Fix broken internal/external links; 301 or remove - **Site architecture**: Logical hierarchy; pages within 3–4 clicks from homepage - **Orphan pages**: Add internal links to pages with no incoming links - **Pagination**: Prefer pagination over infinite scroll for crawlability - **Crawl budget**: Reduce waste on duplicates, redirects, low-value URLs (see below) - **AI crawler optimization**: SSR for critical content; URL management; reduce 404/redirect waste (see below) ## Initial Assessment **Check for project context first:** If `.claude/project-context.md` or `.cursor/project-context.md` exists, read it for site structure. Identify: 1. **Site structure**: Flat vs. deep hierarchy 2. **Framework**: Next.js, static, SPA, etc. 3. **Key paths**: Sitemap, robots.txt, API, static assets ## Best Practices ### Redirect Chains & Loops - Fix multi-hop redirects; point directly to final URL - Loops: URLs redirecting back to themselves; break the cycle ### Broken Links (4xx) - Fix broken int...