feat(web): full SEO stack — metadata, JSON-LD, sitemap, robots, OG image
Some checks failed
Deploy to Production / deploy (push) Failing after 46s
Some checks failed
Deploy to Production / deploy (push) Failing after 46s
Ported and adapted from the BuildMyDiscord SEO setup: - lib/seo.ts — single source for site constants, the FAQ data (shared by the rendered FAQ and the FAQPage schema so they never drift) and JSON-LD builders. - Rich root metadata: title template, keywords, Open Graph, Twitter card, robots directives, canonical. - JSON-LD: Organization + WebSite + SoftwareApplication sitewide, FAQPage on the landing page. No AggregateRating — there are no real reviews yet. - app/robots.ts — allow all, explicit allow-list for AI answer-engine crawlers (GPTBot, ClaudeBot, PerplexityBot, …), disallow private routes. - app/sitemap.ts — every public marketing + docs route. - app/opengraph-image.tsx — monochrome on-brand 1200x630 share card. - app/manifest.ts + public/llms.txt. - Per-page metadata for pricing, changelog, security, privacy, terms, docs, templates and status. - opengraph-image + apple-icon pinned to the edge runtime — next/og crashes during a Node-runtime prerender. Verified: next build passes; /robots.txt, /sitemap.xml, /manifest.webmanifest and /opengraph-image all generate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
38
apps/web/app/robots.ts
Normal file
38
apps/web/app/robots.ts
Normal file
@@ -0,0 +1,38 @@
|
||||
import { SITE_URL } from '@/lib/seo';
|
||||
import type { MetadataRoute } from 'next';
|
||||
|
||||
// AI search/answer crawlers — explicitly allowed so the product surfaces in
|
||||
// ChatGPT, Claude, Perplexity, Google AI and similar answer engines.
|
||||
const AI_CRAWLERS = [
|
||||
'GPTBot',
|
||||
'OAI-SearchBot',
|
||||
'ChatGPT-User',
|
||||
'ClaudeBot',
|
||||
'anthropic-ai',
|
||||
'Claude-Web',
|
||||
'PerplexityBot',
|
||||
'Perplexity-User',
|
||||
'Google-Extended',
|
||||
'CCBot',
|
||||
'Applebot-Extended',
|
||||
'Amazonbot',
|
||||
'meta-externalagent',
|
||||
'DuckAssistBot',
|
||||
];
|
||||
|
||||
export default function robots(): MetadataRoute.Robots {
|
||||
return {
|
||||
rules: [
|
||||
{
|
||||
userAgent: '*',
|
||||
allow: '/',
|
||||
disallow: ['/api/', '/admin', '/dashboard', '/login'],
|
||||
},
|
||||
// Aggressive scraper with no search value.
|
||||
{ userAgent: 'Bytespider', disallow: '/' },
|
||||
{ userAgent: AI_CRAWLERS, allow: '/', disallow: ['/api/', '/admin', '/dashboard'] },
|
||||
],
|
||||
sitemap: `${SITE_URL}/sitemap.xml`,
|
||||
host: SITE_URL,
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user