Add robots.txt generation and sitemap chunking

- bunext sitemap now writes public/robots.txt (configurable via the
  robots config key; robots: false or --no-robots to skip). Hand-written
  robots.txt files are never overwritten without a robots config
- Large sitemaps are split into public/sitemap-chunks/sitemap-N.xml
  (sitemap.chunk_size, default 10,000, max 50,000; 50MB per file) with
  sitemap.xml as a sitemap index; stale chunks are cleaned up
- Export bunextGenerateRobotsTxt
- Run tests in publish.sh

Co-Authored-By: Claude Opus 5.5 <[email protected]>

CLAUDE.md

@@ -19,7 +19,7 @@ bun run dev         # tsc --watch
 bunext dev          # Start dev server with HMR and file watcher
 bunext build        # Bundle all pages for production
 bunext start        # Start production server from pre-built artifacts
-bunext sitemap      # Write public/sitemap.xml using config.sitemap
+bunext sitemap      # Write public/sitemap.xml and public/robots.txt using config.sitemap / config.robots
 ```
 
 ## Architecture