Routing Bot Traffic
EdgeComet works in the request path: your CDN or proxy detects bot requests and routes them to EdgeComet, which answers with rendered, cached HTML. Human visitors are unaffected - their requests never pass through EdgeComet.
The dashboard's Configuration - Integration page walks you through the setup for your platform and tracks each step's status.
Supported platforms
- Cloudflare Workers - the fastest path. Connect your Cloudflare account and the guided wizard sets up the routing worker for you automatically.
- Fastly VCL
- AWS Lambda@Edge
- Nginx
- Other platforms - anything that can branch on the requesting user agent can route bots to EdgeComet; the integration page provides the details your setup needs.
The setup steps
- Allow EdgeComet to reach your origin. EdgeComet fetches pages from your site to render and cache them, so its requests must not be blocked by your firewall or bot protection. The integration page lists what to allow.
- Route bot traffic. Install the routing rule on your CDN or proxy. For Cloudflare this is automatic; for the other platforms the integration page provides the configuration.
- Edge SEO snippet (optional). Independent of the routing step: a small snippet that applies your Edge SEO title and meta-description changes to real users in the browser, so bots and humans see the same values. Install it whenever your team is ready - it is not required for bot-side operation.
Once bot traffic flows, continue to verifying the integration.